-- Logs begin at Tue 2022-08-30 15:37:49 UTC, end at Tue 2022-08-30 16:43:10 UTC. -- Aug 30 16:03:06.335722 dv-cinder-852208-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 16:03:06.345852 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 16:03:06 2022] *** Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: nodename: dv-cinder-852208-13-iscsi Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: machine: x86_64 Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: clock source: unix Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: pcre jit disabled Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: detected number of CPU cores: 4 Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: current working directory: / Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: detected binary path: /usr/bin/uwsgi-core Aug 30 16:03:06.347988 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: your processes number limit is 47680 Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: your memory page size is 4096 bytes Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: detected max file descriptor number: 2048 Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: lock engine: pthread robust mutexes Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: thunder lock: enabled Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 16:03:06.348578 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 16:03:06.387484 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: Python main interpreter initialized at 0x56285a56e9c0 Aug 30 16:03:06.387484 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: python threads support enabled Aug 30 16:03:06.387484 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: your server socket listen backlog is limited to 100 connections Aug 30 16:03:06.387484 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: your mercy for graceful operations on workers is 90 seconds Aug 30 16:03:06.387823 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 16:03:06.387903 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: *** Operational MODE: preforking *** Aug 30 16:03:06.388167 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: *** uWSGI is running in multiple interpreter mode *** Aug 30 16:03:06.388167 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: spawned uWSGI master process (pid: 81593) Aug 30 16:03:06.388371 dv-cinder-852208-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 16:03:06.389431 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 1 (pid: 81605, cores: 1) Aug 30 16:03:06.389431 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 2 (pid: 81606, cores: 1) Aug 30 16:03:06.389431 dv-cinder-852208-13-iscsi devstack@keystone.service[81593]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 16:03:10.697911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 16:03:10.697911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Aug 30 16:03:10.698705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 16:03:10.698705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Aug 30 16:03:10.702465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 16:03:10.702465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Aug 30 16:03:10.702994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 16:03:10.702994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Aug 30 16:03:10.749655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 16:03:10.749871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 16:03:10.750027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] command line args: [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 16:03:10.750294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 16:03:10.750473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ================================================================================ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 16:03:10.750730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.750980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_dir = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.751171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.751380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_source = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.751457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 16:03:10.751565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] debug = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.751658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 16:03:10.751767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.752025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] command line args: [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 16:03:10.752094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 16:03:10.752157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.752225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ================================================================================ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 16:03:10.753209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_dir = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_source = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_config_append = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] debug = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.753936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_dir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_config_append = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.754883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.755106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.755187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.755354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_dir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.755354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756748 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.756950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.757971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] publish_errors = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.758868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] standard_threads = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] publish_errors = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.759891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_journal = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_json = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_stderr = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.760905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_syslog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] standard_threads = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.761602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.761809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.761945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_journal = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.762020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.762123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_json = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.762201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.762299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_stderr = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.762454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.762532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_syslog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.762696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 16:03:10.762777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.762945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.763957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.764932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.765994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.766981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.767974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.768957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.769891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.770837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.771989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772228 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.772924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.773960 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.774894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.775852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.776982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.777955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.778960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.779830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.780832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.781950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.782847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.783935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.784994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.785893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.786933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.787973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.external = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.788882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.password = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.token = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.external = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.789788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.password = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.token = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.790906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.791919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.792916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.793904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.794903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.795957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.796977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.797991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.798995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.799963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.800948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.801930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.802057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.802179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.802793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.802913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.803934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.804906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.805919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.806936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.807990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.808882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.809873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.810996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.811875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.812947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.813951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.814923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.815883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.816897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.817796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.818472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.818602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.818729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.818851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.818932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.819881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.820999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.821886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.822901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.823920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.824878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.825851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.826918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.827994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.828991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.829872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.830897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.831915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.832901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.833948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.834954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.835981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.836872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.837883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.838904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.839898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.840893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.841885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 16:03:10.842385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 16:03:10.868168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.868475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.868606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.869088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.869325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.869558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.877135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.878061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.906266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.906510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.906968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.907203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.961114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:10.961397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.011521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.011832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.012233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.012472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.167761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.168021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.168209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.168434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.168651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.168871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.169455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.169696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.169920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.170145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.170372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.170591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.191958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.192178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.192401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.192595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.192727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.193980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.194104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.194269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.194486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.204958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.205583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.217978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.218982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.219923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 16:03:11.236905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.237109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 16:03:11.237455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.237660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 16:03:11.237941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.238209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.238470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.238732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.239581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.239885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 16:03:11.240239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.240443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 16:03:11.240723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.240990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.241528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.241795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.248448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.248736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.249013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.249305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.251192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.251500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.251775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.252064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.256855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.259605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.259791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.260076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.260364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.260653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.260952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.261330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.261479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.261834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.262020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.262301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.262503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.262882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.263104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.263104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.263600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.263727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.263871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.264149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.264496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.264826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.265152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.265426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.285348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.287380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.287675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.288581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.290060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.290351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.292245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.292535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.292824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.295026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.295341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.295633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.299340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.299632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.301944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.302825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.304333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.304689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.304916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.305201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.305480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.307037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.307359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.307650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.307731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.307935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.308029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.308214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.308295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.308532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.308805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.309085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.323731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.324015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.326713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.327042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.327855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.331239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.331336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.331411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.334352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.334649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.335558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.338640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.338858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.341723 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.342028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.342129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.342319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.342607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.342892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.343484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.345282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.345586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.345873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.346166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.346456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.346738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.359809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.360367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.360648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.360919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.363421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.363717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.364009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.364294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.367739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.369417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.370971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.371267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.371402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.372948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.373908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.374196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.374483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.374565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.374763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.374853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.375054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.377445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.377735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.378140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.378432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.378733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.387193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.387514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.387799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.388082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.388367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.388648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.388935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.390769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.391061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.391480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.391769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.392062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.392342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.392632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.404438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.407493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.408065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.410374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.410986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.411868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.413374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.413647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.413928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.414009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.415461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.417065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.417351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.417649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.420540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.421962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.424145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.424882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.425179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.425470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.425663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.425846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.429202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.429497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.429788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.430073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.434211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.434506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.434790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.437896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.438186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.438466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.441719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.442021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.442586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.442934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.443247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.443535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.443816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.444105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.444385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.444660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.444925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.445194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.445469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.445562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 16:03:11.445759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.446043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.446321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.446602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.446872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.447148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.447452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.447742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.448016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.448289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.448554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.448820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.449109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 16:03:11.480238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56285a56e9c0 pid: 81605 (default app) Aug 30 16:03:11.483836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206bd2-0993-4a13-af39-b0ca7e1ba895 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:11.484114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56285a56e9c0 pid: 81606 (default app) Aug 30 16:03:11.484261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206bd2-0993-4a13-af39-b0ca7e1ba895 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:11.484424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206bd2-0993-4a13-af39-b0ca7e1ba895 None None] PATH_INFO: `/v3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:11.488882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Tue Aug 30 16:03:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:18.132548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:18.133070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:18.133174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:18.138875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:18.144233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:18.144772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:18.144772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:18.216115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: INFO dbcounter [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] Registered counter for database keystone Aug 30 16:03:18.235546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG oslo_db.sqlalchemy.engines [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 16:03:18.242231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writer thread running {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 16:03:18.422856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81605) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 16:03:18.423280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81605) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 16:03:18.481909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:18.483533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:18.613353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 16:03:18.907685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:18.976331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: INFO dbcounter [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] Registered counter for database keystone Aug 30 16:03:18.997783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG oslo_db.sqlalchemy.engines [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 16:03:19.003879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writer thread running {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 16:03:19.124077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab7980-2146-4866-aa7d-11cc8c3ea526 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:19.128356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:19.128975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:19.129266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:19.129524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:19.138181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:19.189033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 16:03:19.189311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dc55c-9655-4e82-bd35-f0563870e0d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:19.198968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Aug 30 16:03:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:03:19.204953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:19.220306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883ce6-e630-457f-b1aa-fce5b163b701 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:19.225417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:19.226519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:19.226809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:19.227094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] PATH_INFO: `/v3/projects/82db1e68be334ddf84a8e69d95cda33e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:19.228700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] RBAC: Authorizing `identity:get_project(project_id=82db1e68be334ddf84a8e69d95cda33e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:19.281498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 16:03:19.281839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b31a3-c8ae-4cf2-80e2-d4f5de6822b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:19.287199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Tue Aug 30 16:03:19 2022] GET /identity/v3/projects/82db1e68be334ddf84a8e69d95cda33e => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:21.200979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede14-9427-44d0-915e-1a0793321e0b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:21.201568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede14-9427-44d0-915e-1a0793321e0b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:21.201568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede14-9427-44d0-915e-1a0793321e0b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:21.205655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:21.211903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:21.212349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:21.212460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:21.232891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:21.234413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:21.243042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 16:03:21.512394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ede14-9427-44d0-915e-1a0793321e0b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:21.525049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ede14-9427-44d0-915e-1a0793321e0b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:21.529313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:21.529907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:21.530191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:21.530472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:21.538712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:21.540512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e9c6-8cdf-4899-8309-50034e37b53f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:21.549214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Aug 30 16:03:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:03:21.554704 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:21.563812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b9660-3d5c-4407-a4c8-969832198a65 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:21.568674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:21.569293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:21.569578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:21.569852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:21.571160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:21.573014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572296-08e9-413a-af03-2bc25e79553e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:21.574526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Tue Aug 30 16:03:21 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:23.567923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b840-6978-4c96-bc5c-4a6d287ce11b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:23.568465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b840-6978-4c96-bc5c-4a6d287ce11b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:23.568465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b840-6978-4c96-bc5c-4a6d287ce11b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:23.569793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:23.575027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:23.575472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:23.575622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:23.587724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094dc52-c167-494b-a928-5a99ebe675f6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:23.588011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094dc52-c167-494b-a928-5a99ebe675f6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:23.588291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094dc52-c167-494b-a928-5a99ebe675f6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:23.589586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:23.594415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:23.594705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:23.594995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:23.595317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:23.595629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:23.637780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81606) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 16:03:23.638185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81606) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 16:03:23.681356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:23.687665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:23.689253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:23.795506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:23.949292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:23.976375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.059243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcafde7-d322-43d8-ae86-a0c223241928 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.062614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.092617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.093249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.093532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.093908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.105757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:24.115486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:24.116466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:24.117681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:24.122441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.166455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea4d42-9999-436f-9883-7dfa9322a923 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.168766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.204188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.204782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.204975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.205261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.205545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.224882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2fa-d016-4964-aa95-62801be4cf02 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.258439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.259368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.259646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.260328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:24.260640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.262058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.273850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.274817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:24.275494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.280331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.374162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:03:24.380106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.383859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ab7d-8dfe-4222-84dc-c2fce6ab718d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.384146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.417396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.418023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.418305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.418578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.419737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.421198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.422767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:24.428805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.443567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.481758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc133d-511f-4790-91fb-02a8b95a8294 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.516486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.517330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.517613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.517886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.528718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e5c5-deab-444d-9e9d-b5492fb1ef43 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.536878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.538370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:03:24.557236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:24.558174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:03:24.559199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:03:24.559199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:03:24.559199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:24.559199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:03:24.564165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.567149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.567909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.568188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.568458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.569570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.571052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.572784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:24.578441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:24.624264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.662883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:24.670047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cbe2c-7a1b-4593-ab3e-9defff9e66dc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.683262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26cd8-b4ff-41e5-bdc9-e023bc83ec30 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:24.706905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.707546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.707830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.708106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.709298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:24.709605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.710988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518c2bf-6a56-441a-a2e7-599088d22c43 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.720413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:24.721023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:24.721301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:24.722074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:24.722811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.domain.id=default, user.id=b24720c8535f4d608668a1e3b4a4419c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:24.723122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:24.724651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e190c68-1b44-4d41-b830-71209a57aab1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:24.728592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:24.731379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Aug 30 16:03:24 2022] GET /identity/v3/role_assignments?user.id=b24720c8535f4d608668a1e3b4a4419c&scope.domain.id=default&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:26.936779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c504-4722-4739-8c91-b2d6cb3c18d0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:26.937319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c504-4722-4739-8c91-b2d6cb3c18d0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:26.937319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c504-4722-4739-8c91-b2d6cb3c18d0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:26.938948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:26.943871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ad84-f198-4d29-99f7-28fe77d32712 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:26.944741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ad84-f198-4d29-99f7-28fe77d32712 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:26.945240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ad84-f198-4d29-99f7-28fe77d32712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:26.972234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d211ad84-f198-4d29-99f7-28fe77d32712 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:26.974286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d211ad84-f198-4d29-99f7-28fe77d32712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:27.025817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d9263-e671-4fab-ad0b-935ea47cc3a2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.026516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d9263-e671-4fab-ad0b-935ea47cc3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.027001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d9263-e671-4fab-ad0b-935ea47cc3a2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.028550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 12/23] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:27.033782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8c94-45ba-4d24-bd3e-bb13324e09a8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.034321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8c94-45ba-4d24-bd3e-bb13324e09a8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.034791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8c94-45ba-4d24-bd3e-bb13324e09a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.065832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fc6a8c94-45ba-4d24-bd3e-bb13324e09a8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:27.072516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc6a8c94-45ba-4d24-bd3e-bb13324e09a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:27.113157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 12/24] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:27.173121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 13/25] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:27.366950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ebb59-7d26-4f53-8e00-26084a5fe478 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.367633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ebb59-7d26-4f53-8e00-26084a5fe478 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.368121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ebb59-7d26-4f53-8e00-26084a5fe478 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.369929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 13/26] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:27.374932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.375248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.375539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.400207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:27.401478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:27.463302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:27.501897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 14/27] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:27.527187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ebb59-7d26-4f53-8e00-26084a5fe478 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:27.650455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:27.690977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ebb59-7d26-4f53-8e00-26084a5fe478 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:27.728320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:27.737409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:27.738052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.738331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.738603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.750420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:27.756376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:27.757237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:27.758021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 14/28] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:27.763791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:27.879816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006af10-d5d8-4f02-911e-e129c611571d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:27.920540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:27.921424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.921834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.922210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.923024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:27.923468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:27.924462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c1b12-3fa9-41e8-b15f-61588d170715 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:27.925079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6197b3-2fec-40fc-b463-0e7fa8d412c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:27.926685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:27.936508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 15/29] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:27.937671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63851f-c1e7-4540-b6fc-36e523709d9d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.938178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63851f-c1e7-4540-b6fc-36e523709d9d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.938532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63851f-c1e7-4540-b6fc-36e523709d9d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.943255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 16/30] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:27.943255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63851f-c1e7-4540-b6fc-36e523709d9d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:27.970683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:27.971565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:27.971848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:27.972124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:27.983499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:27.984977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:03:28.001828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:28.002701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:03:28.003518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:03:28.003518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:03:28.003518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:28.003518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:28.004142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.047433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:28.061950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63851f-c1e7-4540-b6fc-36e523709d9d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.102759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.103471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.103893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.104173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.104962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.106372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.137603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba27c-209f-45e4-86eb-cee0107f2360 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.173824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.174382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.174670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.174952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.176484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.178497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb1fd-a2f5-49be-8820-4a33472075b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.184923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 16/32] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:28 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:28.186153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.186444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.186725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.206291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:28.207571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:28.245606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 17/33] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 30 16:03:27 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:28.246728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.311170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 17/34] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:28.311911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.316534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:28.369179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:28.376290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d648e2-43b5-424f-929f-4d70795b7cae None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.420831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.421522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.421816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.422097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.422834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:28.423159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.425553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.448234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 18/35] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:28.456284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.535817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb162a80-454a-4f9b-be49-56c3892a1083 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.576169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.576810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.577096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.577374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.578509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.580012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85357f9-2364-4749-add0-bd8c428266e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.584812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 18/36] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:28.586064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd497fa-22b6-4b94-9729-1e7b9e18325d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.586355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd497fa-22b6-4b94-9729-1e7b9e18325d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.586638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd497fa-22b6-4b94-9729-1e7b9e18325d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.592220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 19/37] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:28.593883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa164-896c-45bd-97cb-4029d25ef5e1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.594730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd497fa-22b6-4b94-9729-1e7b9e18325d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.616673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:28.648219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.649432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.649871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.650634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.651691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.653510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88813d1-e9e9-4208-afc5-1f5c1858697c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.694859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 19/38] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:03:28 2022] POST /identity/v3/projects => generated 301 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:28.696620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.697139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.697642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.722192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:28.727504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:28.729237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:28.735981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd497fa-22b6-4b94-9729-1e7b9e18325d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:28.778961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:28.780230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.780714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.781247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] PATH_INFO: `/v3/domains/default/users/b24720c8535f4d608668a1e3b4a4419c/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.783698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b24720c8535f4d608668a1e3b4a4419c, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:28.785426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80aa3-89f3-42d7-a34e-c3af8d5665a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:28.824359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 20/39] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:28.825271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:28.914564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 20/40] 10.222.0.57 () {58 vars in 1361 bytes} [Tue Aug 30 16:03:28 2022] PUT /identity/v3/domains/default/users/b24720c8535f4d608668a1e3b4a4419c/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:03:28.917013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f499a-7af3-42aa-94d7-b413284a5554 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.917597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f499a-7af3-42aa-94d7-b413284a5554 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.918077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f499a-7af3-42aa-94d7-b413284a5554 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.923288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 21/41] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:28.929772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:28.930407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:28.930916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:28.958029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:28.960142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:29.066882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 22/42] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:29.075274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:29.078175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb31ae-56e3-4ef7-ade4-ebdbd3fa32a0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:29.123151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:29.123808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:29.124087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:29.124367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:29.125126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:29.126596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:29.138260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:29.149091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 21/43] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 30 16:03:28 2022] POST /identity/v3/roles => generated 225 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:29.246102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:29.265105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9ae38-5e75-4a9b-b375-1c483decd694 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:29.303237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:29.303879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:29.304161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:29.304434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:29.305618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:29.307189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:29.308965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 23/44] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:29.314488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:29.360481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:29.463287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cdac9-9f23-4aa3-b729-d511e03e24d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:29.488558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6984dc-c1d3-4e63-b0fb-82c4d86538c9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:29.502919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:29.503512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:29.503801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:29.504378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:29.504756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:29.506815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f38db-2c40-410b-b23b-709b5f36feb0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:29.526415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:29.527913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:29.528182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:29.528487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:29.529203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:29.530532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270ff7c-3704-45b6-ba24-7ec2229f0734 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:29.547256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 22/45] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:03:29 2022] POST /identity/v3/projects => generated 312 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:29.561189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 30 16:03:29 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:30.940699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643e96-367b-4c2d-a971-64d0575e7672 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:30.941220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643e96-367b-4c2d-a971-64d0575e7672 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:30.941334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643e96-367b-4c2d-a971-64d0575e7672 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:30.942758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 23/47] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:30.948099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d802-e189-4f22-aa26-fb058d176f34 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:30.948514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d802-e189-4f22-aa26-fb058d176f34 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:30.948776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d802-e189-4f22-aa26-fb058d176f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:30.967257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-59e0d802-e189-4f22-aa26-fb058d176f34 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:30.968437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59e0d802-e189-4f22-aa26-fb058d176f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:30.975967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:31.054876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 25/48] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:31.322003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb643e96-367b-4c2d-a971-64d0575e7672 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.336136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:31.426328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb643e96-367b-4c2d-a971-64d0575e7672 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:31.460439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:31.461070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.461352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.461628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.471788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:31.476861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:31.477744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:31.478494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 24/49] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:31.482068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca808f-c932-4721-8ff0-060ad37731e3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.482434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca808f-c932-4721-8ff0-060ad37731e3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.482775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca808f-c932-4721-8ff0-060ad37731e3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.484068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.484162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 26/50] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:31.489421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.489705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.489982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.508130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:31.509372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:31.556130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:31.576537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 27/51] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:31.587955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527b550-b566-4322-b4eb-afa016f56665 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:31.620928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:31.621533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.621820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.622095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.622765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:31.623084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:31.624538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:31.633855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 25/52] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:31.639517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.709741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:31.740326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dda778-507b-4ec3-bd8c-9b9af61b1d11 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:31.774278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:31.774915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.775203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.775517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.776661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:31.778119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:31.779736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 28/53] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:31.785335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.843078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.889444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6387b0-2af8-43e1-b64a-501b20c7e67e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:31.920076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:31.925918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:31.926489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.927383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.927665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.928878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:31.930350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:31.932157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 26/54] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:31.937863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:31.947562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a170-08f6-4e25-8b34-5d5adb3aab1d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:31.983644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:31.984267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:31.984544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:31.984818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:31.986203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:31.987692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:31.989781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 29/55] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:31.995322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:32.006351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:32.048431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d72e97-3c36-49b5-8b65-524dead48bfd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:32.082900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:32.083561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:32.083849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:32.084127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:32.084863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.domain.id=default, user.id=b24720c8535f4d608668a1e3b4a4419c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:32.085175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:32.086620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e87801-44e1-4d5e-9464-e5dbad716823 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:32.092438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 27/56] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Aug 30 16:03:31 2022] GET /identity/v3/role_assignments?user.id=b24720c8535f4d608668a1e3b4a4419c&scope.domain.id=default&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:32.095740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43858a0e-bc3d-4a51-a136-8a912f93cab1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:32.240018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:32.240635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:32.240916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:32.241193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:32.241847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:32.243312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0612cf-104a-4d1d-b284-bf4a0611cc7d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:32.275483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 30/57] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Aug 30 16:03:31 2022] POST /identity/v3/projects => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:03:34.228019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa96be-78ad-4bd3-a24f-f84777324800 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:34.228544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa96be-78ad-4bd3-a24f-f84777324800 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:34.228632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa96be-78ad-4bd3-a24f-f84777324800 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:34.229915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 28/58] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:34.234987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:34.235664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:34.235891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:34.254249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:34.256074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:34.311205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:34.323084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 31/59] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:34.596925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa96be-78ad-4bd3-a24f-f84777324800 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:34.661047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:34.699869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa96be-78ad-4bd3-a24f-f84777324800 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:34.733624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:34.734262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:34.734551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:34.734829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:34.736261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:34.737731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c752b8-314b-4f0a-98e6-9d0eee82bc3b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:34.739390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 29/60] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:34.745035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:34.843943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b856a2d-cc4f-4bc8-be1a-44aa39367904 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:34.877085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:34.877650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:34.877948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:34.878242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:34.878994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:34.880633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876f4e6-83c2-4a6d-9dd3-a100af3475ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:34.908068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 32/61] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:03:34 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:03:37.865988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af51eb-ddc1-4350-8669-7a394e883e92 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:37.866497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af51eb-ddc1-4350-8669-7a394e883e92 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:37.866594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af51eb-ddc1-4350-8669-7a394e883e92 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:37.867915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 30/62] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:37.874280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf273ee-e28c-4e8b-917f-f5274a38417b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:37.875480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf273ee-e28c-4e8b-917f-f5274a38417b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:37.876080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf273ee-e28c-4e8b-917f-f5274a38417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:37.911146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cdf273ee-e28c-4e8b-917f-f5274a38417b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:37.912409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cdf273ee-e28c-4e8b-917f-f5274a38417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:37.973351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:38.003739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 33/63] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:38.023187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbf302-4133-4a51-9b7f-ee1a0e11b5d6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.023892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbf302-4133-4a51-9b7f-ee1a0e11b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.024418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbf302-4133-4a51-9b7f-ee1a0e11b5d6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.026199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 31/64] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:38.032477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679763c-e13b-4d8c-8000-c5c841ca7cc2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.032772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679763c-e13b-4d8c-8000-c5c841ca7cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.033056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679763c-e13b-4d8c-8000-c5c841ca7cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.060891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9679763c-e13b-4d8c-8000-c5c841ca7cc2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:38.062094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9679763c-e13b-4d8c-8000-c5c841ca7cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:38.162407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 34/65] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:38.384778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef51b-fe95-4f85-868b-180c21a0db0d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.385438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef51b-fe95-4f85-868b-180c21a0db0d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.385914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef51b-fe95-4f85-868b-180c21a0db0d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.387811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 32/66] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:38.392819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.393126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.393416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.403865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97dd1-0dde-4c47-b980-91c14d57de53 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.404153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97dd1-0dde-4c47-b980-91c14d57de53 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.404436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97dd1-0dde-4c47-b980-91c14d57de53 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.406226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 33/67] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:38.411501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.411791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.412106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.419242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:38.420543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:38.436857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:38.438101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:38.501381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 35/68] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:38.502254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:38.527690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 34/69] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:38.554719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:38.564857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:38.660935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:38.666177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cc3a5-984b-4135-b9c8-3b375e3147c3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:38.705566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:38.707005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.707529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.708897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.733120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:38.740340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:38.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:38.742333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 36/70] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:38.750013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8210f75-8df6-4426-b70a-055c35339e75 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:38.753093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:38.795922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:38.796560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.796841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.797115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.808100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4feff425-5e6f-4393-9c52-5fea04fbe282 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:38.813077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:38.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:38.815112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 35/71] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:03:38.816667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4104-ed52-4b8e-b11a-7b974dee0e7e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.816667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4104-ed52-4b8e-b11a-7b974dee0e7e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.816667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4104-ed52-4b8e-b11a-7b974dee0e7e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.818490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 36/72] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:38.821316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f4104-ed52-4b8e-b11a-7b974dee0e7e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:38.886083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:38.893536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb998c35-16ea-4b2f-a207-88e97e30c355 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:38.927706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:38.928310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.928589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.928865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.929529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:38.929833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:38.931309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7f643-4e32-4192-8722-2c0b20e5cc11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:38.931815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f4104-ed52-4b8e-b11a-7b974dee0e7e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:38.944665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 37/73] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:38.945897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.946187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.946462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.959770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:38.967952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:38.969168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:38.973450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:38.974059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:38.974347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:38.974628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:38.975338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:38.975649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:38.977057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:38.985932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 37/74] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:38.986735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.036989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 38/75] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:39.037862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.093300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130d2a-2111-446f-910e-e4551b15a5ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.133493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.134413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.134871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.135341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.136113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.149160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:39.149442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16e0a5-b1e7-4c19-a817-74245041eb1f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:39.155056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:39.156056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:39.156924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:39.158869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 38/76] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:03:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:39.160019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.195191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.195953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.196306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.196690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.207060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:39.211210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:39.212635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:39.213487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:39.214244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 39/77] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:39.215310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.277538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865793cc-bd39-4805-8e27-b96e97292598 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.313140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.313778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.314063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.314341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.328529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.329993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.331705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 39/78] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:39.332548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.340869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d3fa5-f555-4c81-a25e-6c072b0a6b31 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.377459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.378094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.378376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.378654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.379888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.381352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.382931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 40/79] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:39.388228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.407642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.434801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9c5b-5b94-4ff9-b023-64b2d4cd67a0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.436533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.475181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.475746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.476026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.476312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.476993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:39.477302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.478697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.487273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 40/80] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:39.487893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.500457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc0a6b-5cb9-40c9-b7e6-90607d6d9aca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.537635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.538251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.538538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.538821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.539521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:39.539832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.541209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.550126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 41/81] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:39.551405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.593146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbf7b1-5105-421f-972a-9579b2734913 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.630761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.631436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.631729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.632006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.633141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.634602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.635182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.636329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 41/82] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:39.637134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.663020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7c75d-1baa-4690-a3c7-402b0250f4be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.670675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.698524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.699207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.699545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.699825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.744687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0156-c570-40e0-bdb5-469f961eae19 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.770927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.772436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.782231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:39.782848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:39.783136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:39.783448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:39.850043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.857198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:39.859011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:39.873998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 42/83] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:39.874856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:39.904110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:39.975165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc325295-3f1f-4e6e-a86a-d47b03a5a59b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:39.976086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 42/84] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:39.976942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.014464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.014987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.015304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.015588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.016736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.018218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.019865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 43/85] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:40.021156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.067819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.085103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1491-d118-4c6d-be28-09e6d02b5a90 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.116881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.122091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.122700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ad8a7-cf8f-4a59-90d5-942362274c24 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.122803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.123005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.123315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.124458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.125924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.127545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 43/86] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:39 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:40.128311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.159012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.159655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.159933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.160212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.160896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.162421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.184596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 44/87] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:03:40 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:40.185807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.244174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad675-3e23-4b72-afbe-f68677381bc6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.289349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.290608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.291098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.291747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.293542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.295791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.298073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 44/88] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:40.299761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.309019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea4e8-0088-493d-ae74-febf97d6e17a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.310736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.352412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.353123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.353341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.354045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.354892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.356451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.359017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 45/89] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:40.359017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.391542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.439399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa70372-bee2-4e65-9339-3c1b2967cc99 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.483999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.485041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.485691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.486161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.499505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804299af-e747-4282-97c9-01f08de8bd00 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.545984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.546437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.546558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.546829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.576325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.577817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.599597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.633537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.635024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.649040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.686106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 45/90] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:40.687113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.745292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 46/91] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:40.746172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.793120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3982c-3f97-4c24-9ea4-e399d506a17c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.827616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.833590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:40.834207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:40.834957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:40.835270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:40.836013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:40.836326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:40.838467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:40.847604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:40.849150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:40.892306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6a252-6ef9-4579-99c5-ad689f909b58 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:40.908645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:40.998958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:41.000291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.000731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.001181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.011249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b24720c8535f4d608668a1e3b4a4419c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:41.011249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:41.011249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:41.011583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c04fb-bc2a-4e57-8c94-97a488974df7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:41.021427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/role_assignments?user.id=b24720c8535f4d608668a1e3b4a4419c&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:41.022181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:41.053084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:41.054149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.054442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.054717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.056520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:41.057998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:41.060550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 47/94] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:40 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:41.068516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:41.189112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365620-7401-4800-81df-fe0c86225db4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:41.191527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:41.228424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:41.229062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.229346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.229622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.230470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:41.230784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:41.232297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:41.238707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 48/95] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:41 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=1927f695a15842babcb8a7b304466dbb => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:41.273281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ec2ab-b0cb-46ee-8c01-6aa4a2b32ae6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:41.335356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:41.336025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.336315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.336600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] PATH_INFO: `/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.344156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e0ff16d39eb442958eaec8f2e453c6fa)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:41.345969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81251-f30d-4b4e-afa9-09bc98d518a7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:41.348000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 48/96] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:41 2022] GET /identity/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa => generated 312 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:41.355008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:41.562597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca99f8-9bb4-49d4-96de-567d1d9d8b54 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:41.631269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:41.631269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:41.631269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:41.631269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:41.631667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=e0ff16d39eb442958eaec8f2e453c6fa, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:41.631667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:41.631667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164d8ca-725d-4516-9e98-b4bbb9f78308 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:41.637540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 49/97] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:41 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=e0ff16d39eb442958eaec8f2e453c6fa&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:42.465436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efe63f-814d-455c-a726-c899f33994a7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:42.465946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efe63f-814d-455c-a726-c899f33994a7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:42.466019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efe63f-814d-455c-a726-c899f33994a7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:42.467410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 49/98] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:42.473033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21c17-afb1-4e1d-8fe9-964d41c7d558 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:42.473961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21c17-afb1-4e1d-8fe9-964d41c7d558 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:42.474418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21c17-afb1-4e1d-8fe9-964d41c7d558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:42.504265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6a21c17-afb1-4e1d-8fe9-964d41c7d558 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:42.506348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6a21c17-afb1-4e1d-8fe9-964d41c7d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:42.541301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:42.605505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 50/99] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:43.148444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efe63f-814d-455c-a726-c899f33994a7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:43.151856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:43.240785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0f199-edfa-45d9-95be-23d0d541861f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.241414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0f199-edfa-45d9-95be-23d0d541861f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.241889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0f199-edfa-45d9-95be-23d0d541861f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.243576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 51/100] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:43.250825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b42f8-0e87-4c6e-9ed7-29b51c34ed35 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.251465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b42f8-0e87-4c6e-9ed7-29b51c34ed35 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.251959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b42f8-0e87-4c6e-9ed7-29b51c34ed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.267538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efe63f-814d-455c-a726-c899f33994a7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:43.283444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3b1b42f8-0e87-4c6e-9ed7-29b51c34ed35 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:43.284929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3b1b42f8-0e87-4c6e-9ed7-29b51c34ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:43.306327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:43.306966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.307267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.307546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.327628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:43.335937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:43.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:43.337634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:43.339302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 50/101] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:43.344058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:43.415173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 52/102] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:43.450659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:43.485291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7551f07-dcb7-4fb7-b863-6c28c2f3c09d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:43.520668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:43.521284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.521568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.521845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.522517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:43.522825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:43.524243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:43.529563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aa32-bd06-44ff-88a3-c8bf10424a49 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.530432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aa32-bd06-44ff-88a3-c8bf10424a49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.530926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aa32-bd06-44ff-88a3-c8bf10424a49 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.532498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 53/103] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:43.535266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 51/104] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:43.538916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:43.539457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.539966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.540434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.558801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:43.560358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:43.693701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 54/105] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:43.720841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d229-f5fa-4c58-bc4f-ffe0061eed65 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:43.755892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:43.766817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:43.767489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:43.767774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:43.768054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] PATH_INFO: `/v3/users/5fc6916d4a534121994a409d16a46cc4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:43.793262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] RBAC: Authorizing `identity:get_user(user_id=5fc6916d4a534121994a409d16a46cc4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:43.794729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:43.796506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 52/106] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:43 2022] GET /identity/v3/users/5fc6916d4a534121994a409d16a46cc4 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:43.802229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:43.825965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:43.874090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:43.950863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38f81-2bc4-4dbc-9926-ffdbe26be5a5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.013304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.014073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.014364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.014644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.062037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba3ab-8b29-4f23-9ef1-13e9abe44947 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.127461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.127942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.128723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.129118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.129503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.130384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.135982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.137599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.139830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 53/107] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:44.140834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.162171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.242468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.312699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 55/108] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:44.313987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.347055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6245915-c005-43d6-926c-2ce05bba5764 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.408617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.409638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.409979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.410467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.421474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:44.427075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:44.428019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:44.428770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:44.430656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 54/109] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:44.431681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.434976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02614a-d325-43f2-aa68-8441f1c9ce0a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.455098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.474285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.474900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.475184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.475488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.476147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.477608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12526bda-59d0-4f43-a5eb-89ff30838385 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.508226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 56/110] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:03:44 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:44.509768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeeea2-9eb8-40e2-9546-a3bed8b24426 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.509987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeeea2-9eb8-40e2-9546-a3bed8b24426 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeeea2-9eb8-40e2-9546-a3bed8b24426 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.511735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 57/111] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:44.512286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeeea2-9eb8-40e2-9546-a3bed8b24426 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.571910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2fb61-df6c-4bad-924e-ef8f8bb830a7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.590155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.625959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.626851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.627426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.628662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.630529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.632476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e931f-00ab-46b4-bcf9-2e4601502c81 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.639937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 55/112] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:44 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:44.639937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654f2-45fb-491d-bd83-174b7b3581fd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.639937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654f2-45fb-491d-bd83-174b7b3581fd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.639937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654f2-45fb-491d-bd83-174b7b3581fd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.642740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 56/113] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:44.644494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d720b-8f6b-43c4-bcd8-d6dc3aae5183 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.645024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d720b-8f6b-43c4-bcd8-d6dc3aae5183 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.645997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d720b-8f6b-43c4-bcd8-d6dc3aae5183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.659695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeeea2-9eb8-40e2-9546-a3bed8b24426 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.670151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f34d720b-8f6b-43c4-bcd8-d6dc3aae5183 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:44.671790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f34d720b-8f6b-43c4-bcd8-d6dc3aae5183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:44.702864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.703837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.704147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.704488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.705112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:44.705421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.706817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.721351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 58/114] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:44.722480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.754677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 57/115] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:44.756152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2dbb0-7e68-4bef-b29a-4ffdf8b3c0e1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.756331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2dbb0-7e68-4bef-b29a-4ffdf8b3c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.756612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2dbb0-7e68-4bef-b29a-4ffdf8b3c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.760540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.780917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0ca2dbb0-7e68-4bef-b29a-4ffdf8b3c0e1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:44.782288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0ca2dbb0-7e68-4bef-b29a-4ffdf8b3c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:44.854523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566d97-320f-46dd-8b73-e8286e0abc93 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:44.869525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 58/116] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:44.871257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e94cb-93e3-4fad-96d2-3741493c8754 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.871729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e94cb-93e3-4fad-96d2-3741493c8754 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.872178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e94cb-93e3-4fad-96d2-3741493c8754 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.873661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 59/117] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:44.874601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e94cb-93e3-4fad-96d2-3741493c8754 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:44.902562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:44.903523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.903967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.904462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.905922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=5fc6916d4a534121994a409d16a46cc4) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:44.906402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:44.908638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7eafd-4229-4ba6-bf72-2fbd94f224b3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:44.917696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 59/118] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:44 2022] GET /identity/v3/role_assignments?user.id=5fc6916d4a534121994a409d16a46cc4&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:44.927249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:44.927249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:44.927249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:44.937632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:44.964377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:44.965616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:45.059964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e94cb-93e3-4fad-96d2-3741493c8754 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.100110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.100808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 60/119] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:45.101409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.101960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.102464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.108797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:45.111785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:45.113782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 60/120] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:44 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:45.120260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.164523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.189823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.266237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49bc3-f50d-4f9f-be7b-96826a1b4764 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.306376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94330720-f65b-4f2e-9872-2b36ed9b3387 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.315140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.320945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.321712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.321990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.322258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.323518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:45.324992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:45.326746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 61/121] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:45.332048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.355095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.356605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.357404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.358113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.371340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:45.376627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:45.377500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:45.378309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:45.381138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 61/122] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:45.382144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.497797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eadb4-ab4b-4731-9bfa-52ba039e7494 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.520695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.527520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ad70d-194e-4450-bb61-c2804c11bae5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.540526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.541626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.542172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.542650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b3a51bbf235046feb98976f941a96c39/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.545172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, user_id=b3a51bbf235046feb98976f941a96c39, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:45.546942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:45.568389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 62/123] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:45 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b3a51bbf235046feb98976f941a96c39/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:45.568547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.569326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.569660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.569947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.570371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.583813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:45.588968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:45.589927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:45.590686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 62/124] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:45.591857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.612689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.713408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b806b-e734-4514-b833-2d66dc15d2f9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.719584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961a8f8-1f4c-4337-b6d2-08d5444e8510 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.755809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.756453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.756738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.757070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.768992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:45.769672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:45.770648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:45.771134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:45.771642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:45.772527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:45.773055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:45.774205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:45.775061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:45.775873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 63/125] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:45.776525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.777696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:45.789350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 63/126] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:45.790943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:45.799776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.908001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:45.928714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea43674-65fc-409d-842e-89a4a7c98336 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:45.941885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bc27b-d309-4545-b2f1-492c6bcb49ad None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.001972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.002589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.002874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.003150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.003852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:46.004162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.006239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.007938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.008556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.008837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.009251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.009789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:46.010093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.011508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.015296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 64/127] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.016228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.024562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:03:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.025405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.124402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e56fe-08b3-4281-b00f-012cb839532c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.160448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.161064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.161355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.161633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.162823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.164349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.165985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 65/129] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.166788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.175581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.187243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.190669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bc9c3-999c-4e06-8da3-fe81dc967db4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.229898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.231001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.231556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.232039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.233456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.235188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.237414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.238569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.275890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3303fe-9786-4a67-b741-d2d28f04b8cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.312372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.313013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.313299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.313583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.314739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.316293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.318556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 66/131] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.319468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.364893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99527b-b3a2-4d16-b79b-b71a97f16c88 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.435707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.451625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a64ffb-bb54-49dd-bf9e-be495ebfc711 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.460069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.460735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.461023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.461302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.468323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.470050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.472005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 66/132] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.472825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.492348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.493103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.493322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.493602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] PATH_INFO: `/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.497204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] RBAC: Authorizing `identity:get_project(project_id=e0ff16d39eb442958eaec8f2e453c6fa)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.499505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.502696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 67/133] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.504151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.506250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.663936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b40b-30b2-479f-ac3c-27aa3707c3d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.701186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4535-4d54-42a2-9cfa-76bba2d59b12 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.738693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.740206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.741731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.742385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.744645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.748905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.751043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 67/134] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:46 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:46.756174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.777664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.780851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.781316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.781763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b24720c8535f4d608668a1e3b4a4419c/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.785536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, user_id=b24720c8535f4d608668a1e3b4a4419c, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.788342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:46.806933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 68/135] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:46 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b24720c8535f4d608668a1e3b4a4419c/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:46.808420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:46.829627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.940431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74448e-d8f8-4cec-a0ca-be8f20d7cab4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.941456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:46.980206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4434-9ae2-46f0-ab1e-12bec9326175 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:46.981135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:46.982019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:46.982520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:46.983060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b3a51bbf235046feb98976f941a96c39/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:46.985849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, user_id=b3a51bbf235046feb98976f941a96c39, role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:46.987594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2c49-aaae-4627-8ca9-40d3f0541dd6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:47.008086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 69/136] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:46 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/b3a51bbf235046feb98976f941a96c39/roles/1927f695a15842babcb8a7b304466dbb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:47.046061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:47.047692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.047979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.048261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] PATH_INFO: `/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa/users/b3a51bbf235046feb98976f941a96c39/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.051408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0ff16d39eb442958eaec8f2e453c6fa, user_id=b3a51bbf235046feb98976f941a96c39, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:47.053429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d327ea4-2acc-4308-af69-646ecbfbed05 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:47.074981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 68/137] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:46 2022] PUT /identity/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa/users/b3a51bbf235046feb98976f941a96c39/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:47.143534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c365ad-db94-4aa0-95cc-b0a4ff1b50f0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.144048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c365ad-db94-4aa0-95cc-b0a4ff1b50f0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c365ad-db94-4aa0-95cc-b0a4ff1b50f0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.146184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 70/138] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:47.153052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab6bf-2fe1-4341-8dde-f628463ba455 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.153345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab6bf-2fe1-4341-8dde-f628463ba455 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.153629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab6bf-2fe1-4341-8dde-f628463ba455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.174802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70dab6bf-2fe1-4341-8dde-f628463ba455 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:47.176120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70dab6bf-2fe1-4341-8dde-f628463ba455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:47.256934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 69/139] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:47.486949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6720c6-7ae4-4cb1-8751-d848539a24bd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.487363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6720c6-7ae4-4cb1-8751-d848539a24bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.487560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6720c6-7ae4-4cb1-8751-d848539a24bd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.489437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 71/140] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:47.494691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77593695-347b-4862-bc76-91991715627f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.494988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77593695-347b-4862-bc76-91991715627f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.495329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77593695-347b-4862-bc76-91991715627f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.520963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-77593695-347b-4862-bc76-91991715627f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:47.522100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77593695-347b-4862-bc76-91991715627f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:47.560024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6720c6-7ae4-4cb1-8751-d848539a24bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:47.609130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:47.616391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 70/141] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:47.695591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:47.736751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6720c6-7ae4-4cb1-8751-d848539a24bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:47.772453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:47.773010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.773295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.773571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.774700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:47.776182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:47.782330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 72/142] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:47.783730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77593695-347b-4862-bc76-91991715627f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:47.893976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77593695-347b-4862-bc76-91991715627f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:47.932553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:47.933170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:47.933453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:47.933741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:47.934407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:47.935889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:47.965325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 71/143] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:03:47 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:48.069973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:48.196204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350e5ef-562c-4232-a708-19aa53fa0fe2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:48.241343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:48.241984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:48.242269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:48.242559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:48.256347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:48.261524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:48.262369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:48.263110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:48.268663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:48.329088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:48.378307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8bd0-51a1-471b-baa7-91ca757d391a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:48.415783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546d03-2314-482c-9197-c839e3632877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:48.416405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546d03-2314-482c-9197-c839e3632877 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:48.416687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546d03-2314-482c-9197-c839e3632877 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:48.416963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546d03-2314-482c-9197-c839e3632877 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:48.417693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44546d03-2314-482c-9197-c839e3632877 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:48.418003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44546d03-2314-482c-9197-c839e3632877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:48.419470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44546d03-2314-482c-9197-c839e3632877 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:48.428982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:48.434481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:48.444569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:48.589491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c44dd-46cc-410d-affd-5449fdd9013b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:48.639947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:48.640714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:48.643271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:48.643271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] PATH_INFO: `/v3/users/5fc6916d4a534121994a409d16a46cc4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:48.644341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] RBAC: Authorizing `identity:get_user(user_id=5fc6916d4a534121994a409d16a46cc4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:48.646002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:48.647816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:48 2022] GET /identity/v3/users/5fc6916d4a534121994a409d16a46cc4 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:48.653327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546d03-2314-482c-9197-c839e3632877 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:48.793938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546d03-2314-482c-9197-c839e3632877 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:48.811948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:48.833546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:48.834194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:48.834479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:48.834758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:48.835927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:48.837409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143984e-b76b-4111-9276-d3f39e48c6ee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:48.839455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 73/147] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:48 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:48.844907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:48.963574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:49.001470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e987c-b5df-4198-9a97-bb5bcafacb3e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.002107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e987c-b5df-4198-9a97-bb5bcafacb3e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.002596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e987c-b5df-4198-9a97-bb5bcafacb3e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:49.004222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 74/148] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:49.007975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f53cf-e5a3-4b11-b7ca-fb544ee8718e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:49.014574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.015172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.015686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:49.046973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:49.048632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:49.068627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:49.069719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.071442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.071442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/5fc6916d4a534121994a409d16a46cc4/roles/396ddbab5a1f4d4695997f3eb5f4e009` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:49.079248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, user_id=5fc6916d4a534121994a409d16a46cc4, role_id=396ddbab5a1f4d4695997f3eb5f4e009)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:49.079248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:49.108680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 75/149] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:48 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/users/5fc6916d4a534121994a409d16a46cc4/roles/396ddbab5a1f4d4695997f3eb5f4e009 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:49.161468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 75/150] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:49.619807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:49.751570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fac9c1-2705-49ef-8acd-0ddedd87d4b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:49.786402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:49.792284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:49.792986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.793386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.793693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:49.806868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4ff8301d-e41f-4690-990a-2beee742460c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:49.811770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:49.812620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:49.813933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 76/151] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:49.819147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:49.939827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110749e-c384-402f-aa4e-c9fc88107b50 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:49.961484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:49.969674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fddebe-eafb-49d2-bc47-46e4cd192a09 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.970166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fddebe-eafb-49d2-bc47-46e4cd192a09 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.970358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fddebe-eafb-49d2-bc47-46e4cd192a09 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:49.972498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 77/152] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:49.978746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:49.979032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:49.979331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.008609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:50.009824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:50.015969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:50.016614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.016891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.017171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.017860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:50.018170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:50.019613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:50.041204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 76/153] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:50.048432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:50.209284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 78/154] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:50.215803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66c8d9-bb72-49fc-952b-502907fff0e7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:50.266030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:50.267044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.267553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.268027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.271985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:50.273969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096d7a0-8e02-4bc2-80df-705268a030ab None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:50.276061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 77/155] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:50 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:50.284416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:50.315356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d427551-a11b-4a24-920f-6ce3e58eebcc None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.315964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d427551-a11b-4a24-920f-6ce3e58eebcc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.316456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d427551-a11b-4a24-920f-6ce3e58eebcc None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.318605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 78/156] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:50.335971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c65699-9981-4596-801d-d946d8563637 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.336604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c65699-9981-4596-801d-d946d8563637 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.337076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c65699-9981-4596-801d-d946d8563637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.364398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-64c65699-9981-4596-801d-d946d8563637 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:50.366032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64c65699-9981-4596-801d-d946d8563637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:50.459743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:50.465220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769ef83-2257-48ab-a349-8c6e02d18f36 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:50.483575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 79/157] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:50.507365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:50.530807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:50.531458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.531749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.532026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.542060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324edde-580b-4b0d-b4f7-35e6c53a46b2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.542355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324edde-580b-4b0d-b4f7-35e6c53a46b2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.542636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324edde-580b-4b0d-b4f7-35e6c53a46b2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.544021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 80/158] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:50.550925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.551806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.552106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.588319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:50.589532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:50.633362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:50.634861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:50.697853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 81/159] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:50.698608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:50.833247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 79/160] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:50.844324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:50.846324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a298e7-2f12-4efb-824c-7fa5d219ba36 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:50.875665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:50.879850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:50.917240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:50.917938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:50.918225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:50.918498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:50.935370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:50.944419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:50.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:50.946185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 82/161] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:50.954263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:50.992463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d88c-d314-4feb-b989-b7613fac77db None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.079938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.080591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.080872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.081146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.088018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:51.089451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:51.091144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 80/162] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:50 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:51.100828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.107436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8181f-4597-40c2-b81a-8333b61ef40f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.164557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fffd4-6465-4012-be31-9409b436465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.165179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fffd4-6465-4012-be31-9409b436465d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.165516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fffd4-6465-4012-be31-9409b436465d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.165795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fffd4-6465-4012-be31-9409b436465d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.166528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fffd4-6465-4012-be31-9409b436465d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:51.166838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fffd4-6465-4012-be31-9409b436465d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:51.168268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fffd4-6465-4012-be31-9409b436465d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:51.181264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 83/163] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:51.187418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fffd4-6465-4012-be31-9409b436465d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.225253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.265625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.294166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff990-3600-4140-b5ac-30a02590ed60 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.332162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fffd4-6465-4012-be31-9409b436465d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.369476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.370034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.370373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.370646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] PATH_INFO: `/v3/users/b24720c8535f4d608668a1e3b4a4419c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.371818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] RBAC: Authorizing `identity:get_user(user_id=b24720c8535f4d608668a1e3b4a4419c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:51.373276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:51.374912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 84/164] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/users/b24720c8535f4d608668a1e3b4a4419c => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:51.375781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.376428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.377052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.377336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.377610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.378476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:51.378796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:51.380460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:51.397762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 81/165] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:51.398739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.479592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.486030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16e98-d01b-4170-9355-9f30e90545a2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.526150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.526948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.527332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.527680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.539176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:51.544700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:51.546610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 85/166] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:51.549021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.653255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb356929-39df-45c9-9f5e-e75175f9d7f9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.657427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f584c-6d25-40f7-ae33-b5cc84fb9802 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.694379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.695006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.695331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.695608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.727627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.773979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:51.775678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:51.780638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:51.781314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:51.781588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:51.781861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:51.815840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:51.821807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:51.822701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:51.823590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 82/167] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:51.824250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.862198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.885955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 86/168] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:51.886751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:51.947503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:51.991069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5aff1-e4b6-4eaf-8d36-b669d64743b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:51.993699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead2276-f329-456b-9f19-92e4c2da51bf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.031168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.031819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.032098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.032366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.033038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:52.033338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.034824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.044605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 87/169] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.045590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.053063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.053734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.053969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.054249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.054926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:52.055250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.056666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.070988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 83/170] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.071885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.156594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77bc67-ef3e-45e2-9b62-7304873081c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.187617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:52.198366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.198933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.199244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.199521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.200717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.202209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.204222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 88/171] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.205128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.243268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:52.297511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0d7b5-0cf6-4f5a-948a-94f2c7372569 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.326117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9e2b1-d9c7-4106-9f8e-c9c8f8f62e66 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.365678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.366317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.366602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.366877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.368056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.369503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.371095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 89/172] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.372039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.373561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.374220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.374611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.374894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] PATH_INFO: `/v3/users/b3a51bbf235046feb98976f941a96c39` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.376388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] RBAC: Authorizing `identity:get_user(user_id=b3a51bbf235046feb98976f941a96c39)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.377857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fe42-d055-4d2d-9c68-e34863b920ea None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.379531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 84/173] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/users/b3a51bbf235046feb98976f941a96c39 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.389004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a215d-d13e-4a6e-8406-8d8119e1cdba None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.389295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a215d-d13e-4a6e-8406-8d8119e1cdba None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.389577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a215d-d13e-4a6e-8406-8d8119e1cdba None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.390863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 85/174] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:52.391719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a215d-d13e-4a6e-8406-8d8119e1cdba None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.435843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:52.489710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdea92-4212-4074-ac58-de962884a7fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.529301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b24720c8535f4d608668a1e3b4a4419c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.535243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.541775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 90/175] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/role_assignments?user.id=b24720c8535f4d608668a1e3b4a4419c&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:52.543176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:52.666726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363284ce-4648-410a-bdf4-ac09e2f9ca34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.667891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a215d-d13e-4a6e-8406-8d8119e1cdba None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:52.703754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:52.715383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.716154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.716445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.717059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.764141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:52.764687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.764971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.765246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.784852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:52.804833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.806950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64207b73-c43e-44fa-a01d-6dce4075f986 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.862083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:52.862083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506604aa-2056-4bf1-a12d-b03987ee2363 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:52.959629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 91/176] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:52.960909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.961199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.961477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.987907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:52.989009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 86/177] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:52.989440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:52.990423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc06e-ccc7-4444-8833-79e2c1e26abe None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:52.990715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc06e-ccc7-4444-8833-79e2c1e26abe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:52.990994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc06e-ccc7-4444-8833-79e2c1e26abe None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:52.992409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 87/178] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:52.993038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc06e-ccc7-4444-8833-79e2c1e26abe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.009068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.058471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.090977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 92/179] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:53.091765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.131108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc06e-ccc7-4444-8833-79e2c1e26abe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.175688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.176866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.177583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.177866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] PATH_INFO: `/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.179358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] RBAC: Authorizing `identity:get_project(project_id=e0ff16d39eb442958eaec8f2e453c6fa)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:53.181658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80ee8-7eed-4498-b66b-25c6826ca181 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:53.184093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 88/180] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:52 2022] GET /identity/v3/projects/e0ff16d39eb442958eaec8f2e453c6fa => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:53.185503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6c9c-3e51-4768-91f7-4dea771de2d9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.185788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6c9c-3e51-4768-91f7-4dea771de2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.186062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6c9c-3e51-4768-91f7-4dea771de2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.213088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d75f6c9c-3e51-4768-91f7-4dea771de2d9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:53.214306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d75f6c9c-3e51-4768-91f7-4dea771de2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:53.250174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4fab1-db22-4a4c-9172-9e0c599890ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.295201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.295859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.296141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.296411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.297055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 89/181] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:53.298097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27e7a-e66a-4481-bc65-8129838b2aa2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.298389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27e7a-e66a-4481-bc65-8129838b2aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.298667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27e7a-e66a-4481-bc65-8129838b2aa2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.299940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:53.301737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:53.303991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 93/182] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:53.304985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.307842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 90/183] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:53.309921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c81a5-ea97-4977-8c32-4b01fb40ff87 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.309921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c81a5-ea97-4977-8c32-4b01fb40ff87 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.309921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c81a5-ea97-4977-8c32-4b01fb40ff87 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.310891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 91/184] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:53.314989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.314989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.314989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.330116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.334638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:53.336712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:53.393197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.422309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad5e24-b283-40de-8775-3215875edfc4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.425459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 92/185] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:53.432721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.461947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.462920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.463427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.463890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.464821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=e0ff16d39eb442958eaec8f2e453c6fa, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:53.465371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:53.467082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74536f8d-87f2-4604-b427-ac9a13ce99bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:53.474560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 94/186] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=e0ff16d39eb442958eaec8f2e453c6fa&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:53.476129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.476660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.477125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.507738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:53.509334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:53.569724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9d2c2-5c47-4009-8b5b-7a8905a3a448 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.611174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.611916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.612204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.612482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.613229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=b3a51bbf235046feb98976f941a96c39) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:53.613542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:53.615168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:53.622790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 93/187] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/role_assignments?user.id=b3a51bbf235046feb98976f941a96c39&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=1927f695a15842babcb8a7b304466dbb => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:53.623071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.638733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 95/188] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:53.639781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.653782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.702374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.776623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd9121-7fdd-446c-a9ac-82a5cbd875c9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.808008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4903d-eca7-4d69-b841-70b4d53cfa45 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:53.818488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.819134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.820687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.820687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.834380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:53.839418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:53.840321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:53.841168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 96/189] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:53.841833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.859836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:53.861328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:53.861844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:53.862477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:53.878601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:53.886258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:53.887199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 94/190] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:53.888486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:53.944575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:53.984049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4dbe-d08e-4f96-bb6f-b77c05b1f504 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.010342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.017861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d561b-171e-40a2-8590-dd5991b91d75 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.025104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.025728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.026005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.026275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.027407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.028855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.030418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 97/191] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.031175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.052630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.053237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.053527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.053801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.054462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:54.054769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.056163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.064401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 95/192] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.065202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.134653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50923e6a-7803-4c8f-a669-178273f19cda None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.164891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12efd25-861c-47f4-894f-6f4c71dc8ddc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.170461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.171064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.171402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.171677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.172351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:54.172653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.174039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.179745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.187260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 98/193] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.187260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.199811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.200434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.200712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.200992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.210998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:54.216181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:54.217205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 96/194] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:54.218005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.236540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.288859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f731d0d-2882-4e91-9ad5-2110fc1db530 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.317897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479d0a-4cae-4521-8156-d13093754662 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.323103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.323737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.324025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.324298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.324955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.326403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.346798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 99/195] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:03:54 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:54.347672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.363581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.365091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.365841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.366116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] PATH_INFO: `/v3/users/5fc6916d4a534121994a409d16a46cc4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.367752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] RBAC: Authorizing `identity:get_user(user_id=5fc6916d4a534121994a409d16a46cc4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.369867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.376708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 97/196] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/users/5fc6916d4a534121994a409d16a46cc4 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.376708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.415908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.471211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a09cc4-8eee-4f4f-99cc-665adb63a674 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.482448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.502711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71bfa4-8f2c-4911-930d-3e82e80da946 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.518035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.518679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.518959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.519256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.537195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.538684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.540392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 100/197] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.541163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.547918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.548532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.548817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.549094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.549822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:54.550128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.551744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.564959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 98/198] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:54.566707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.669185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f62ab-3d85-471a-9972-20df991bb057 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.685294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.697090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7abb0-9ad0-4278-8e1b-10780040404d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:54.710312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.711024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.711384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.711730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.733532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:54.734146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:54.734432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:54.734710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:54.752552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.792935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.794863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.809995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:54.811791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:54.905325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 101/199] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:54.906972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.922994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:54.934494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 99/200] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:54.939265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:54.986844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.026414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6dbe-3e4e-4585-b9c3-d55350a4a52a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.044034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4df6d-0641-45da-8a99-092419849e15 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.063660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.064626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.065201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.065665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.067084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.069101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.071149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 102/201] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.072216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.085142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.085780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.086070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.086347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.087524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.088970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.090614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 100/202] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:54 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.091409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.184357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.209224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf080e-c040-410b-a727-015f9e02bd00 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.220086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e094d7-7e9e-49fb-820c-cbcd5c65115c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.238283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.253863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.254678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.254964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.255260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.256417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.258098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.260037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 103/203] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.260605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.262988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.263738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.264122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.264499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.336713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.338383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.370342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb3e1e-cae4-4e4e-b007-cb23e0c38967 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.408826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.409433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.409717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.409995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.411436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e, user.id=5fc6916d4a534121994a409d16a46cc4) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:55.411854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.413335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.420114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 104/204] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/role_assignments?user.id=5fc6916d4a534121994a409d16a46cc4&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.420753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.438066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.476188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 101/205] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:03:55.483177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.507392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.546819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0ca4-8e11-4888-80d8-113c0c5864e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.585974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.586608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.586895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.587173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.588010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=d9548f84299b480fbaf3b5f10a9c19f8) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:55.588412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.590218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.597015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 105/206] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/role_assignments?user.id=d9548f84299b480fbaf3b5f10a9c19f8&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.665599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d140a-7f71-4fb4-a084-eb51fe80e426 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.708684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.709258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.709539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.709815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.711129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.712683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caa92a-a66f-4adc-a236-b7eb737bd348 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.714450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 102/207] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:55.720507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:55.808291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:55.831725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b39e12-7fa6-419a-9e2d-220dc3268438 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:55.871837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:55.872448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:55.872734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:55.873008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:55.873740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=d9548f84299b480fbaf3b5f10a9c19f8) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:55.874038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:55.875612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c318a-9142-45e6-94f5-7e97685af1a7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:55.884907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 106/208] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:55 2022] GET /identity/v3/role_assignments?user.id=d9548f84299b480fbaf3b5f10a9c19f8&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=1927f695a15842babcb8a7b304466dbb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:56.583586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5a900-53d2-4454-8ca6-3e0a82d027c1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:56.584658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5a900-53d2-4454-8ca6-3e0a82d027c1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:56.585138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5a900-53d2-4454-8ca6-3e0a82d027c1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:56.586949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 103/209] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:56.592466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:56.593291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:56.593767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:56.603205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c99c67-9e15-4c7d-b72a-f0dfe8b6b3f3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:56.603517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c99c67-9e15-4c7d-b72a-f0dfe8b6b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:56.603784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c99c67-9e15-4c7d-b72a-f0dfe8b6b3f3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:56.605547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 104/210] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:56.610865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:56.611151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:56.611455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:56.615981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:56.617402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:56.630799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:56.632038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:56.702912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 107/211] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:56.715345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 105/212] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:56.991485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.013382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.034301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:57.121775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:57.166685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be456-6dad-45da-90f4-1de242565165 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:57.191847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398decf2-a4ca-4851-a7fa-7b1ec69cb0d3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:57.230402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:57.236378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.237121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.237805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.268404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:57.274580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:57.275552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:03:57.276327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:57.289211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 106/213] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:57.294359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.295777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:57.297066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.297808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.298447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.300344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:57.302675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:57.309517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 108/214] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:03:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:57.315400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.445012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fa94-e65f-4348-bdc6-b0f50dedd83b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:57.446060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:57.490258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:57.490869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.491148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.491440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.492542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:57.494054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f754-a3d4-4e20-8f79-7ce2e0272274 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:57.511096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:57.531859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 109/215] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:03:57 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:03:57.542251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c464-4cd9-4537-95ae-1cd7ff9f3d00 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:57.613489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f803d2-20f7-425b-b187-558844b018de None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.614084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f803d2-20f7-425b-b187-558844b018de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.614084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f803d2-20f7-425b-b187-558844b018de None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.616422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 110/216] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:03:57.620394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:57.621905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.622190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.622517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.623234 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:57.623550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:57.624968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:57.627903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.628201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.628482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.636029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 107/217] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:03:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:57.641421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.664482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:57.666214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:57.760057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 111/218] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:03:57.770161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d49-ee75-44cb-bee7-bcd17d63bdaa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:57.807575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:57.808141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.808427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.808705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] PATH_INFO: `/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.810891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:57.825534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] RBAC: Authorizing `identity:get_user(user_id=be6aa800b9f74e7b93d1f5b3c9cd2e26)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:57.827066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd8c92-f251-4fe1-b9a7-3df70256d37e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:57.829158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 108/219] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:57 2022] GET /identity/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:57.834644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:57.925008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c358-8298-440e-b3a7-3830e8bf1a6c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.925327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c358-8298-440e-b3a7-3830e8bf1a6c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.925571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410c358-8298-440e-b3a7-3830e8bf1a6c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.928407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 109/220] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:03:57.936816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:57.937175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:57.937521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:57.960604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:03:57.962571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:03:57.974889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba46aeb-a03f-4f95-b877-da6e784840f2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.007510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.022622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.023202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.023521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.023795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.056125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 110/221] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:03:58.057315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.107633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.109273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.163603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2763547-c8bb-4d2a-8d05-1dd1e95d2d93 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.188227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.200877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.201886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.202394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.202865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.226563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:58.233811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:58.234693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:03:58.235536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:03:58.242294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 111/222] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:58.245946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 112/223] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:03:58.247166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.251899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.305339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.401664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63ba6-b1ea-40f5-9087-cd7e10b9fbfb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.411971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06633b7-bfae-4f43-901f-5021e959d155 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.448338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.448547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.449251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.449537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.449816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.450119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.450275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.450699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.451636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.451804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:58.452237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.453340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.453904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.455545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 113/224] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:58.456835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.464382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 112/225] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:58.465383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.516579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.576728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7b0b5-989e-4917-b164-7426efeb259a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.593024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.607113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971d5fb-ccee-4f7a-bebf-62e78579a5dd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.617048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.617963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.618467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.618967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.619931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=be6aa800b9f74e7b93d1f5b3c9cd2e26) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:58.620497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.622606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.629912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 113/226] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/role_assignments?user.id=be6aa800b9f74e7b93d1f5b3c9cd2e26&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:58.631052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.671962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.673485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.674186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.677041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.700718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:03:58.709406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:03:58.710381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:03:58.711154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:03:58.717033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 114/227] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:03:58.725452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.840711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70ba5e-aa94-415d-a859-e83c8641ff62 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.865810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:58.888659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.890403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.891102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.892881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.897917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.899993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 114/228] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:58.902047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d156-5835-4484-b19d-e6db41f9409b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:58.905653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:58.968806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:58.970550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:58.972919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:58.973456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:58.974335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:03:58.974837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:58.976680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:58.993068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 115/229] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:58.999790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:59.033792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:59.089378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e154a-06f2-457f-8f29-a910bd254501 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:59.162286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:59.162927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:59.163205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:59.163514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:59.164651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:59.166100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:59.168119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 115/230] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:59.173534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:59.218957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d81016-ab30-4481-838b-93ab18a7a7a9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:59.250164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:59.290468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:59.291165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:59.291567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:59.291839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:59.293108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:59.294614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:59.296514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 116/231] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:03:58 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:03:59.302252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:59.352873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885285f-7012-4117-b416-ac5221474da2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:59.408294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:03:59.428587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995f3ec-2271-475f-a28d-e494d0900cf2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:59.453326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:59.454030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:59.454318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:59.454598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/users/d9548f84299b480fbaf3b5f10a9c19f8/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:59.456776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, user_id=d9548f84299b480fbaf3b5f10a9c19f8, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:59.458799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df028a-bfaa-4072-b94d-c82569861ccb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:59.471378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:59.472541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:59.473108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:59.473729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:59.475372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:59.477088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:59.479394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 117/232] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:03:59 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:03:59.486377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:03:59.500124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 116/233] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:59 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/users/d9548f84299b480fbaf3b5f10a9c19f8/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:03:59.604207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1024-c694-40a3-93a7-c955026b89ae None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:03:59.649726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:03:59.650706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:03:59.651185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:03:59.651757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/users/d9548f84299b480fbaf3b5f10a9c19f8/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:03:59.654297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, user_id=d9548f84299b480fbaf3b5f10a9c19f8, role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:03:59.656163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed8ac7-548b-45d4-a9c9-aa7d6e428b1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:03:59.672523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 118/234] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:03:59 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/users/d9548f84299b480fbaf3b5f10a9c19f8/roles/1927f695a15842babcb8a7b304466dbb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:00.018352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42c1b-4ffa-4c3c-b65e-02247b1334a1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.018862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42c1b-4ffa-4c3c-b65e-02247b1334a1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.018936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42c1b-4ffa-4c3c-b65e-02247b1334a1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.020556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 117/235] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:00.026097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c015-d546-4d89-8f04-0923192e535a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.026633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c015-d546-4d89-8f04-0923192e535a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.027122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c015-d546-4d89-8f04-0923192e535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.053647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-44e1c015-d546-4d89-8f04-0923192e535a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:00.055196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-44e1c015-d546-4d89-8f04-0923192e535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:00.068251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:00.149449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 119/236] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:00.210519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4461e-f773-48b7-938c-9b3b654926ca None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.210868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4461e-f773-48b7-938c-9b3b654926ca None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.211074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4461e-f773-48b7-938c-9b3b654926ca None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.212783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 118/237] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:00.217735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.218288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.218776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.238712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:00.240256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:00.328005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 120/238] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:00.471759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4461e-f773-48b7-938c-9b3b654926ca None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:00.643387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4461e-f773-48b7-938c-9b3b654926ca None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:00.648400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:00.697364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:00.697872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:00.698253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.698535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.698804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:00.715688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:00.716527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:00.717383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:00.720729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 119/239] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:00.722208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:00.836371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960455d9-81b0-4648-a9ad-5196c17140cb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:00.875755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:00.910203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:00.910863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.911142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.911437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.912112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:00.914710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:00.915607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:00.938462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 120/240] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:00.940673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89738e3-5165-4f7b-abae-46d3c4620b2b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:00.943159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:00.981601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:00.985446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:00.986342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.986620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.986891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.988101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:00.989688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa9e3-b3be-4739-99bd-dd4c2d206082 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:00.995166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 121/241] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:00.995166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c089a4-539a-4c87-acd8-a59971fb6c90 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:00.995166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c089a4-539a-4c87-acd8-a59971fb6c90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:00.995166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c089a4-539a-4c87-acd8-a59971fb6c90 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:00.995166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 122/242] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:01.002377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.002709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.002995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.023175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:01.024382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:01.077325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cbfaa-c5db-4290-ad28-a6c23726c2ff None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.097630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 123/243] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:01.099133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.115701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.116343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.116624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.116897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] PATH_INFO: `/v3/users/082e062e80f5433b9a197cee48858184` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.131100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] RBAC: Authorizing `identity:get_user(user_id=082e062e80f5433b9a197cee48858184)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.132746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.134489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 121/244] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:00 2022] GET /identity/v3/users/082e062e80f5433b9a197cee48858184 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:01.140271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.234517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:01.250554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:01.254912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43fb7-b78c-4614-bb05-b965cad8c497 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.291314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.292203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.292626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.293069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.317405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad9c5c-be45-4cf7-b20c-dd40715b9b52 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.364780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.366861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.427936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.428586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.428862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.429115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.429786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.431408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.460824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:01.462046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 124/245] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:01 2022] POST /identity/v3/users => generated 293 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:01.463087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.504872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 122/246] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:04:01.513086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.660471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4dcea-06c3-4adb-accd-3c80ad6d3cbc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.672742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52827b4-3f54-46c2-abcf-e056518ac1fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.700553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:01.703346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.705264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.705656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.705949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.707986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.710248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.711177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.711864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.712148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.712314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 123/247] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:01.712420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.719462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.724660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:01.730153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:01.731053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:01.731874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 125/248] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:01.737749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.795688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:01.835152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d75bb-88be-4bd7-9899-e4b09c8cffa1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.845104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd19cf-3c6a-43b2-ab1e-eb09842f5866 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:01.884007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.884629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.884912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.885187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.886170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:01.886479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.888503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.899364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 126/249] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:01.904858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:01.907212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:01.907849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:01.908131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:01.908409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:01.909165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=082e062e80f5433b9a197cee48858184) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:01.909478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:01.911799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362611-033b-4cd5-a7ad-e67450119bbc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:01.918694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 124/250] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/role_assignments?user.id=082e062e80f5433b9a197cee48858184&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:01.965417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:02.010494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cdb00-b51b-4460-862a-2c0b4ecf0302 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.010919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cdb00-b51b-4460-862a-2c0b4ecf0302 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.011362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cdb00-b51b-4460-862a-2c0b4ecf0302 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.013145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 125/251] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:02.018981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9d74-4e4e-4e95-b9ea-54cf95f23d36 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.019398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9d74-4e4e-4e95-b9ea-54cf95f23d36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.020301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9d74-4e4e-4e95-b9ea-54cf95f23d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.025770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd318-27d7-4129-966b-e67d88df9ba6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.041431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c8fb9d74-4e4e-4e95-b9ea-54cf95f23d36 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:02.043178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c8fb9d74-4e4e-4e95-b9ea-54cf95f23d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:02.065250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:02.066213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.067243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.067665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] PATH_INFO: `/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.070155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=be6aa800b9f74e7b93d1f5b3c9cd2e26)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:02.072053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:02.074248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 127/252] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:01 2022] GET /identity/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:02.080181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:02.128485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 126/253] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:02.130294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae6993-2eb6-4783-b55b-bdaa5cb3fddb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.130581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae6993-2eb6-4783-b55b-bdaa5cb3fddb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.130859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae6993-2eb6-4783-b55b-bdaa5cb3fddb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.132399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 127/254] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:02.138014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea813-aab6-4812-8153-0539362d052d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.138298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea813-aab6-4812-8153-0539362d052d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.138574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea813-aab6-4812-8153-0539362d052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.162105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-50aea813-aab6-4812-8153-0539362d052d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:02.163565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-50aea813-aab6-4812-8153-0539362d052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:02.200695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34adb9a-dfd0-4516-bb92-dca44ef19bc5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.220094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:02.226645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:02.257657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 128/255] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:02.264686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:02.264686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.264686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.264686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.265074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:02.265074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:02.265349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 128/256] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:02 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:02.272055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aea813-aab6-4812-8153-0539362d052d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:02.377950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aea813-aab6-4812-8153-0539362d052d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.414246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:02.414907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.415184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.415482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/users/be6aa800b9f74e7b93d1f5b3c9cd2e26/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.417389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, user_id=be6aa800b9f74e7b93d1f5b3c9cd2e26, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:02.418845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:02.432991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 129/257] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:02 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/users/be6aa800b9f74e7b93d1f5b3c9cd2e26/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:02.466758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:02.593263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f9b31-e8a9-440d-8ddb-475354d03655 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.647435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:02.647984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.648264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.648535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.660343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:02.667467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:02.668892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:02.669968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 129/258] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:02.675569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:02.715737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:02.732424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:02.812550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:02.908564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b4e8-2fee-4117-a9fc-80cc10b77690 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.935723 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0428-4726-4a42-9fd6-614a94be4fd5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:02.994983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:02.995628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:02.995913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:02.996190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:02.996869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:02.997178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:02.998563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.020936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 130/259] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:03.024944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.026840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.031571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.031854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.032128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.059027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:03.064469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:03.065350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:03.066180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 130/260] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:03.075861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.245598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:03.254512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da203-34d0-46e4-a192-5f78664e96bd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:03.306198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.309627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.310173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.310796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.312384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:03.313995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.316334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 131/261] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:03.323924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.339302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7017-114d-4090-b9ac-f5c3389824d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:03.341851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:03.398063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.398679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.398962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.399252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.399935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:03.400239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:03.401645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.411796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 131/262] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:03.416462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.512503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af61c5-3b80-4df4-b544-a68c0cdccd6d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:03.571026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135e89-8bd7-47d7-89d1-25e41a38d72a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:03.583876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.584500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.584786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.585103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.651639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.652285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.652567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.652839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] PATH_INFO: `/v3/users/d9548f84299b480fbaf3b5f10a9c19f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.655068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] RBAC: Authorizing `identity:get_user(user_id=d9548f84299b480fbaf3b5f10a9c19f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:03.656511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.661944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 132/263] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/users/d9548f84299b480fbaf3b5f10a9c19f8 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:03.662078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:03.666871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.717343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:03.718716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.751588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:03.827642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466247d0-278e-4a8f-8340-b5b782c5bb66 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:03.878579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:03.879381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:03.879665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:03.880043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:03.927253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 132/264] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:03.932342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:03.953624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:03.955110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:03.997841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:04.047549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:04.065646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 133/265] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:04:04.071334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:04.120836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632e556-b0eb-4952-9c6d-7433e31880e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:04.193594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:04.194248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.194533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.194808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.200539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:04.202127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:04.203833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 133/266] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:03 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:04.210666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:04.272253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119a112-7f4f-4145-9c11-f5c95ba3c78a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:04.363877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:04.364533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.364817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.365093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.367249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:04.368993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:04.370773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 134/267] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:04 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:04.372349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f73bee-2523-4710-8faa-4e53a60b56e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:04.380769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:04.387688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:04.416518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:04.417141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.417419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.417688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.418592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=d9548f84299b480fbaf3b5f10a9c19f8) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:04.418906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:04.420757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f35a8-a7a1-4121-8c91-d22ad60f9c34 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:04.422736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:04.437336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 134/268] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:04 2022] GET /identity/v3/role_assignments?user.id=d9548f84299b480fbaf3b5f10a9c19f8&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:04.521707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335cca-b53d-4412-b25a-a4f98f685711 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:04.564293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:04.565212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.565673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.566150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.567707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=d9548f84299b480fbaf3b5f10a9c19f8) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:04.568209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:04.569877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e845-d545-4f76-aaf6-a6659eb97d0f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:04.577218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 135/269] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:04 2022] GET /identity/v3/role_assignments?user.id=d9548f84299b480fbaf3b5f10a9c19f8&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=1927f695a15842babcb8a7b304466dbb => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:04.702957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e764a7-6f95-48b8-9015-7b75b6d100c3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.703331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e764a7-6f95-48b8-9015-7b75b6d100c3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.703557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e764a7-6f95-48b8-9015-7b75b6d100c3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.705164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 135/270] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:04.711699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a67a-7bca-431c-9a73-0f1a076c37e0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.712332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a67a-7bca-431c-9a73-0f1a076c37e0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.712809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a67a-7bca-431c-9a73-0f1a076c37e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.740072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c897a67a-7bca-431c-9a73-0f1a076c37e0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:04.741605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c897a67a-7bca-431c-9a73-0f1a076c37e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:04.829187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:04.843932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 136/271] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:04.993634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79396-4a99-42db-b5c2-fca1cf3d66b5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:04.994029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79396-4a99-42db-b5c2-fca1cf3d66b5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:04.994206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79396-4a99-42db-b5c2-fca1cf3d66b5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:04.995838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 136/272] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:05.001523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243b80-c02c-4fca-b903-78e416c7bc5f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.001810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243b80-c02c-4fca-b903-78e416c7bc5f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.002086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243b80-c02c-4fca-b903-78e416c7bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.025495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d3243b80-c02c-4fca-b903-78e416c7bc5f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:05.026793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d3243b80-c02c-4fca-b903-78e416c7bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:05.107343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 137/273] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:05.161185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d8bc-cc3e-4b80-8f09-b7cceb1c0bf3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.161536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d8bc-cc3e-4b80-8f09-b7cceb1c0bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.161744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d8bc-cc3e-4b80-8f09-b7cceb1c0bf3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.163373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 137/274] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:05.168788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.169074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.169349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.189774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:05.190954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:05.221676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3d8bc-cc3e-4b80-8f09-b7cceb1c0bf3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.268869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 138/275] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:05.333871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3d8bc-cc3e-4b80-8f09-b7cceb1c0bf3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.369700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.370346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.370627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.370903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.391618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:05.397669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:05.398563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:05.399969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 138/276] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:05.408370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.408651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.470957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.487860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.522860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef0a6-c633-4431-be28-36a7b3e6b4c6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.532591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92653cda-8a93-4d40-927f-05d63703fd79 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.559764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.560373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.560659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.560934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.561610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:05.561916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:05.563318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe688-7261-470f-93fd-618e8de87655 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:05.567150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.567804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.568085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.568722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.571948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 139/277] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:05.573135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d886-6c32-4e77-bed0-7c8092511006 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.573418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d886-6c32-4e77-bed0-7c8092511006 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.573689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d886-6c32-4e77-bed0-7c8092511006 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.574883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 140/278] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:05.575618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62d886-6c32-4e77-bed0-7c8092511006 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.583840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:05.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:05.590251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 139/279] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:05.591129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.685165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62d886-6c32-4e77-bed0-7c8092511006 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.695692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ca470-d124-4443-9692-1c2cc6c5d6cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.703101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.713567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.720471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.721102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.721381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.721658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.722785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:05.724267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7123f0b-dd5f-4e13-aa71-bec2268f4c7f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:05.725843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 141/280] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:05.726945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.727251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.727525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.732103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.732742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.733026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.733304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] PATH_INFO: `/v3/users/b86743969b9840c9aa520e8444fa2249` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.746853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:05.748130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:05.749050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] RBAC: Authorizing `identity:get_user(user_id=b86743969b9840c9aa520e8444fa2249)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:05.750524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:05.752205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 140/281] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/users/b86743969b9840c9aa520e8444fa2249 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:05.752972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.815665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 142/282] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:05.821689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.856656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2dc81-25a8-44b5-bb36-996f593885b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.894251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.894867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.895149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.895456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.896130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:05.896457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:05.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:05.911724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 141/283] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:05.911724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:05.945058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2f244-410e-4ed4-afc5-ccd489297e21 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:05.952371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.964909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:05.984838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:05.985444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:05.985980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:05.986254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:05.986903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:05.989030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.013635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 143/284] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:05 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:06.014291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.021550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba0132-aa60-4060-afcd-1d13180939bf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.078404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.079448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.079938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.080404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.081419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, user.id=b86743969b9840c9aa520e8444fa2249, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:06.081965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.083921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.090130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 142/285] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 30 16:04:05 2022] GET /identity/v3/role_assignments?user.id=b86743969b9840c9aa520e8444fa2249&scope.system=all&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:06.096169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.164407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d0d4-826c-494f-95aa-4bed8a27f7a6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.219699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.220745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.221024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.221399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] PATH_INFO: `/v3/users/082e062e80f5433b9a197cee48858184` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.223475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] RBAC: Authorizing `identity:get_user(user_id=082e062e80f5433b9a197cee48858184)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.224271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.226382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 144/286] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/users/082e062e80f5433b9a197cee48858184 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:06.231852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.235400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c55a4-1790-4584-94d0-2261d3cb5e70 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.240262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:06.281192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.283327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.283327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.283327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.291395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:06.297530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:06.302930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:06.303962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:06.304880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 143/287] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:06.311432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.382796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-f8e0-4ac1-86fb-f26937bc04ef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.443941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.444585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.444867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.445145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.446304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.447845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.450119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 145/288] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:06.459540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.521285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052287-42a7-4e58-be9d-9c7bf4450817 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.584028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.584649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.584932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.585200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:06.586213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.587777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.598393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 144/289] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:06.603487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.607921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1aac7-5c9d-4c9b-af8b-9633997afa0e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.626066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:06.636502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:06.682729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.683469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.683750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.684022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/users/082e062e80f5433b9a197cee48858184/roles/396ddbab5a1f4d4695997f3eb5f4e009` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.689115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, user_id=082e062e80f5433b9a197cee48858184, role_id=396ddbab5a1f4d4695997f3eb5f4e009)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.690583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.716337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 146/290] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:06 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/users/082e062e80f5433b9a197cee48858184/roles/396ddbab5a1f4d4695997f3eb5f4e009 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:06.778785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d16dc-3f7a-4722-b23b-3ae5d963208f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:06.822763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:06.823534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:06.823821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:06.824102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] PATH_INFO: `/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:06.825467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] RBAC: Authorizing `identity:get_user(user_id=be6aa800b9f74e7b93d1f5b3c9cd2e26)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:06.827137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:06.829280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 145/291] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/users/be6aa800b9f74e7b93d1f5b3c9cd2e26 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:06.835462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:06.974463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c42a5-be7a-4c4e-b147-c9eb78c29633 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:07.013164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:07.013840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:07.014127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:07.014453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:07.053267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:07.110435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:07.112262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:07.241680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 147/292] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:07.247205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:07.354842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:07.367659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15064-b436-42f9-af39-e92ea2444691 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:07.414992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:07.415716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:07.416003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:07.416332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:07.417411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:07.418870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88975db-75a9-4d36-824b-d515440ca824 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:07.420543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 146/293] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:07 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:07.426575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:07.498569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:07.545231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e856-abb9-44de-bbb0-3d6e1f0676da None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:07.585111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:07.585721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:07.586006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:07.586280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:07.587013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=be6aa800b9f74e7b93d1f5b3c9cd2e26) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:07.587488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:07.588993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cdf78-ce60-45ed-8aa8-7bf416d3cccf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:07.595129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 148/294] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:07 2022] GET /identity/v3/role_assignments?user.id=be6aa800b9f74e7b93d1f5b3c9cd2e26&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:08.375124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7688256-6a82-420f-b489-9c121df130fe None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.375712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7688256-6a82-420f-b489-9c121df130fe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.375712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7688256-6a82-420f-b489-9c121df130fe None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.377360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 147/295] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:08.384787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6765-a3ef-406b-b2c1-86f26acec31e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.385220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6765-a3ef-406b-b2c1-86f26acec31e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.385399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6765-a3ef-406b-b2c1-86f26acec31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.404002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a3b6765-a3ef-406b-b2c1-86f26acec31e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:08.405173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a3b6765-a3ef-406b-b2c1-86f26acec31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:08.415865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd7f4-0d44-4c50-9970-ccd4ab9ea544 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.416152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd7f4-0d44-4c50-9970-ccd4ab9ea544 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.416432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd7f4-0d44-4c50-9970-ccd4ab9ea544 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.417765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 148/296] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:08.423417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063af3a-4767-43d4-8a3a-3461ac9a729f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.423705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063af3a-4767-43d4-8a3a-3461ac9a729f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.424387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063af3a-4767-43d4-8a3a-3461ac9a729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.448548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8063af3a-4767-43d4-8a3a-3461ac9a729f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:08.449752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8063af3a-4767-43d4-8a3a-3461ac9a729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:08.491699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 149/297] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:08.539995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 149/298] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:08.611684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c468-e0e9-4e54-97d0-769b0e1f2d48 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.611996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c468-e0e9-4e54-97d0-769b0e1f2d48 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.612237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c468-e0e9-4e54-97d0-769b0e1f2d48 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.614381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 150/299] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:08.619747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.620031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.620302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.644360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:08.646061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:08.718365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 150/300] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:08.760851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8c468-e0e9-4e54-97d0-769b0e1f2d48 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:08.811687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:08.829133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:08.845545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:08.873207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8c468-e0e9-4e54-97d0-769b0e1f2d48 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:08.914941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:08.915783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:08.916168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:08.916567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:08.928059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:08.933156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:08.934193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:08.935178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:08.942294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 151/301] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:08.943374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:08.958182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852699-4149-4eef-a347-eef4bbf52d86 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.006897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.007820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.008262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.008539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.020509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:09.025387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:09.026280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:09.027076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:09.027713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 151/302] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:09.028639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.054796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1b841-3a19-410e-ba60-7378f098e4ec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.080252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.096146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.096746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.097178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.097451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.098632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:09.099058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.100483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.115876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 152/303] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.115876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.116789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.137398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330e8c2-5d10-4506-825e-081a493d91cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.172896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.173425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.173817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.174091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.175205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.177202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.178769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 152/304] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.183540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.238861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d8995-3255-492d-9e31-f64992a63886 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.272540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.273138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.273417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.273688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.274397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:09.274699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.276098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.284049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 153/305] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.284809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.291572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702dc17-9ac5-468b-8f7f-0fe8c6a6bf86 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.327872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.328495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.328773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.329042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] PATH_INFO: `/v3/users/b86743969b9840c9aa520e8444fa2249` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.330173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] RBAC: Authorizing `identity:get_user(user_id=b86743969b9840c9aa520e8444fa2249)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.331707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb04f21-7aec-43fb-94c1-f8ffeb50fe53 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.333559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 153/306] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/users/b86743969b9840c9aa520e8444fa2249 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.333748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.334506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ac76-c7c1-4e9b-9069-0c868cc85ac3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.334830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ac76-c7c1-4e9b-9069-0c868cc85ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.335133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ac76-c7c1-4e9b-9069-0c868cc85ac3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.336598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 154/307] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:09.337148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ac76-c7c1-4e9b-9069-0c868cc85ac3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.355614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.395397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609544ec-5e07-4a66-b498-d23bb26105e4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.428100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.428617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.428895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.429165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] PATH_INFO: `/v3/groups` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.429783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.431242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.441187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ac76-c7c1-4e9b-9069-0c868cc85ac3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.446903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 154/308] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Aug 30 16:04:09 2022] POST /identity/v3/groups => generated 229 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:09.447802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.495416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.496075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.496356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.496622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] PATH_INFO: `/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.515160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] RBAC: Authorizing `identity:get_user(user_id=2b58f8743e064fed8dbf6d21eea6f4f9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.516672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b348a-78d5-41c1-8bcc-e795b884018e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.518786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 155/309] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.522155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.522760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.523682 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.543616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:09.544849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:09.581315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ad1f6-1b1c-4554-92ba-522a3df09c80 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.590638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.629382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.630735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.631272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.631919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] PATH_INFO: `/v3/system/users/b86743969b9840c9aa520e8444fa2249/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.634077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b86743969b9840c9aa520e8444fa2249, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.635639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.639331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 156/310] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:09.640436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:09.644542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:09.648196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 155/311] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Aug 30 16:04:09 2022] PUT /identity/v3/system/users/b86743969b9840c9aa520e8444fa2249/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:09.778989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473cf82-4f1c-4d7d-830d-733d3bf71429 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:09.818749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:09.819428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:09.819713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:09.819988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:09.820722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, user.id=2b58f8743e064fed8dbf6d21eea6f4f9, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:09.821031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:09.822554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:09.828285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 157/312] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/role_assignments?user.id=2b58f8743e064fed8dbf6d21eea6f4f9&scope.system=all&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:09.919811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.033972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fe36f-99c8-47ce-bb58-d396a2194c64 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.070207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.070842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.071122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.071427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.083131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:10.088470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:10.089332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:10.090146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 156/313] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:10.095656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.176651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:10.218295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d3fb-3ae6-4474-86c0-278a6d18f9e9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.255472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.256087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.256371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.256647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.257389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:10.257700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:10.259079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:10.268405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 158/314] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:10.274560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.308436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:10.385241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d844-301d-44a9-907a-f296bcf16033 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.422441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.423095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.423381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.423656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] PATH_INFO: `/v3/users/082e062e80f5433b9a197cee48858184` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.425002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] RBAC: Authorizing `identity:get_user(user_id=082e062e80f5433b9a197cee48858184)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:10.426525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:10.433014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 157/315] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/users/082e062e80f5433b9a197cee48858184 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:10.434379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.546731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e135c-18dc-41d7-bfe9-9cf738719535 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.580888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:10.586730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.587459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.587960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.588239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.669496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:10.671043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:10.779587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 159/316] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:10.788862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:10.902243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:10.906791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c39cc-d476-463b-87d1-364489da5745 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:10.942429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:10.943431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:10.943951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:10.944425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:10.945811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:10.947509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7e4e5-7987-49ea-8a07-bf65b207bbe1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:10.949708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 158/317] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:10.959804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:11.012892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:11.140106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12b43-85dd-495a-8057-48d71595c600 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:11.201312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:11.202352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.202857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.203404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.204902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, scope.project.id=a95448be579f4092840ecbee1e109310, user.id=082e062e80f5433b9a197cee48858184) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:11.205476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:11.208347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7dae-9634-4986-a8ff-317973944980 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:11.219460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 160/318] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:10 2022] GET /identity/v3/role_assignments?user.id=082e062e80f5433b9a197cee48858184&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:11.603266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bee2-b2d8-41b3-babc-11bd173e52c8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.603781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bee2-b2d8-41b3-babc-11bd173e52c8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.603781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bee2-b2d8-41b3-babc-11bd173e52c8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.605171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 159/319] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:11.610589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84152-3e3a-44a1-98c5-ebfca0473940 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.611306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84152-3e3a-44a1-98c5-ebfca0473940 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.611480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84152-3e3a-44a1-98c5-ebfca0473940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.631712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-17c84152-3e3a-44a1-98c5-ebfca0473940 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:11.632925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17c84152-3e3a-44a1-98c5-ebfca0473940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:11.702485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c36-b588-4f83-bef0-c06a3ab004c3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.702816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c36-b588-4f83-bef0-c06a3ab004c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.703035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c36-b588-4f83-bef0-c06a3ab004c3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.704401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 160/320] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:11.709534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 161/321] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:11.710442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc442a-8b76-449b-885a-84d323b91f36 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.710784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc442a-8b76-449b-885a-84d323b91f36 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.711065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc442a-8b76-449b-885a-84d323b91f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.730897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-59cc442a-8b76-449b-885a-84d323b91f36 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:11.732139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59cc442a-8b76-449b-885a-84d323b91f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:11.804426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 161/322] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:11.881321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329092da-7e15-4b3f-89d9-ae794adce98d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.881613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329092da-7e15-4b3f-89d9-ae794adce98d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.881825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329092da-7e15-4b3f-89d9-ae794adce98d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.883660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 162/323] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:11.890994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a91e-c000-4074-9103-c5715114222a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:11.891324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a91e-c000-4074-9103-c5715114222a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:11.891614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768a91e-c000-4074-9103-c5715114222a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:11.909748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3768a91e-c000-4074-9103-c5715114222a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:11.911515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3768a91e-c000-4074-9103-c5715114222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:11.978457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329092da-7e15-4b3f-89d9-ae794adce98d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:11.983578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 162/324] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:12.007412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.076053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768a91e-c000-4074-9103-c5715114222a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.095365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329092da-7e15-4b3f-89d9-ae794adce98d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.105891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.134655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.135196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.135504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.135777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.136874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.138313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.140311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 163/325] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:12.145695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.184323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768a91e-c000-4074-9103-c5715114222a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.222011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.223012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.223349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.223672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.236734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:12.242130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:12.243034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:12.243835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 163/326] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:12.250510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.256249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.265314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f93c42-fc14-4de1-a3bc-d31323d5f5bc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.299956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.300571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.300866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.301136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] PATH_INFO: `/v3/groups` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.301772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.303207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.320532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 164/327] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Aug 30 16:04:12 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:04:12.321569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.341515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.364629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a822b-5b8c-4d51-a155-3353ab38018d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.409032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.409654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.409938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.410259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.410947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:12.411280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.412703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.426928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 164/328] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:12.428270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.445276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a262-5d00-4caa-81d1-690167f133cc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.490931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.490931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.491402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.491801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.503283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:12.509494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:12.510442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:12.511804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 165/329] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:12.517806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.546581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.548965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8bb67-d25a-4e79-8c9e-3be5c853f1cb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.605875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.606610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.606881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.607475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] PATH_INFO: `/v3/users/b86743969b9840c9aa520e8444fa2249` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.609833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] RBAC: Authorizing `identity:get_user(user_id=b86743969b9840c9aa520e8444fa2249)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.610687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.612992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 165/330] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/users/b86743969b9840c9aa520e8444fa2249 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:12.623383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.648800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.686168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1f035-30b2-41a9-8e45-47a4c8cce2d1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.737805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.738462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.738748 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.739026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.739740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:12.740049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.741534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.752458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 166/331] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:12.757037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93afbe-270f-4413-a96e-b4b5aec52eaa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.758237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:12.809690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.810335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.810721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.810891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.811677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] RBAC: Adding query filter params (role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, user.id=b86743969b9840c9aa520e8444fa2249, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:12.811994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.813503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.829103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 166/332] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/role_assignments?user.id=b86743969b9840c9aa520e8444fa2249&scope.system=all&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:12.851667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:12.897088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76f7d3-bfd0-48f2-8e5d-fb906aa65b7d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:12.942607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:12.943166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:12.943480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:12.943754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] PATH_INFO: `/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:12.944931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] RBAC: Authorizing `identity:get_user(user_id=2b58f8743e064fed8dbf6d21eea6f4f9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:12.946401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bb4a2-816e-4c16-bb7d-f965bf2562d3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:12.948354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 167/333] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:12 2022] GET /identity/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:12.954265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:13.045075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:13.073552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bf52-a1fa-4f6d-bc58-d4814a251610 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:13.109264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:13.109980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:13.110339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:13.110621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] PATH_INFO: `/v3/system/users/2b58f8743e064fed8dbf6d21eea6f4f9/roles/396ddbab5a1f4d4695997f3eb5f4e009` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:13.112356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b58f8743e064fed8dbf6d21eea6f4f9, role_id=396ddbab5a1f4d4695997f3eb5f4e009)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:13.114345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb8212-dc35-4344-bac5-fe0d14add3e4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:13.123806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 167/334] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Aug 30 16:04:12 2022] PUT /identity/v3/system/users/2b58f8743e064fed8dbf6d21eea6f4f9/roles/396ddbab5a1f4d4695997f3eb5f4e009 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:14.732656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af9c27-0a9b-4d31-a454-227772e9672c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:14.733162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af9c27-0a9b-4d31-a454-227772e9672c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:14.733236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af9c27-0a9b-4d31-a454-227772e9672c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:14.734971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 168/335] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:14.742793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407be3ed-4b57-4be0-ae3f-3f4e8f882009 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:14.743261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407be3ed-4b57-4be0-ae3f-3f4e8f882009 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:14.743446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407be3ed-4b57-4be0-ae3f-3f4e8f882009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:14.772292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-407be3ed-4b57-4be0-ae3f-3f4e8f882009 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:14.773513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-407be3ed-4b57-4be0-ae3f-3f4e8f882009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:14.848463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 168/336] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:15.157313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3708-2d38-4146-8240-5b5e7d5308b2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.158022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3708-2d38-4146-8240-5b5e7d5308b2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.158512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3708-2d38-4146-8240-5b5e7d5308b2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.160132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 169/337] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:15.165682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.166221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.166670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.188297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:15.189780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:15.247590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2e3c-0a5c-49ef-b3f2-ba7800fbb9b4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.247958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2e3c-0a5c-49ef-b3f2-ba7800fbb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.248318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2e3c-0a5c-49ef-b3f2-ba7800fbb9b4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.249888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 170/338] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:15.255415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.255792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.256150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.287628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:15.288070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:15.289295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:15.290538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 169/339] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:15.346057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:15.430951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 171/340] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:15.552306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d0534-5d1b-4636-b7e2-56e9227dee08 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:15.593411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:15.594434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.594930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.595435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.608247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:15.613814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:15.614695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:15.615517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:15.618154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 170/341] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:15.628531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:15.712727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:15.733374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:15.795120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74bf34-952a-4c40-893d-68c5eee99ae0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:15.859092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:15.892967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:15.893603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.893887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.894164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:15.896156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:15.896156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:15.897175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:15.914864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 172/342] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:15.920645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:15.942847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda44f4-f562-45c9-a16d-29eea3efaf89 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:15.986701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:15.987755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:15.988271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:15.988756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.001898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9e93da1d-752e-415e-835e-ff4f3a29b251 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:16.009355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:16.010287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:16.011074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:16.012874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 171/343] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:16.014620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832d87-61cb-4e15-a2a3-37678958cd85 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.014903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832d87-61cb-4e15-a2a3-37678958cd85 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.015176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832d87-61cb-4e15-a2a3-37678958cd85 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.021222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 172/344] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:16.021222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832d87-61cb-4e15-a2a3-37678958cd85 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.076227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b2cdf-c09a-4a42-b824-c5b1747179be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.122233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.123492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.123792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.124071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.128373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.162088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832d87-61cb-4e15-a2a3-37678958cd85 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.185167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.206934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.207733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.207912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.208155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.220592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:16.223531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.225302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1dd97-96de-449b-b2e3-48367e773e2d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:16.225858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:16.226757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:16.227674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:16.228869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 173/345] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:16.229713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.356536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfb11f-273e-4acc-80ac-1038b100d1c3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.364678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 173/346] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:04:16.366275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.366761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.367248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.389891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:16.391232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:16.406613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.407819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.408741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.409243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.410933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:16.411467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.413676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:16.422942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.426959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 174/347] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:16.428682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.490288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 174/348] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:16.493210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.498639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.576813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61b6b6-4ddb-4e85-a3bf-57c7dbec5a4a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.619162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb940c-5ad3-4ee0-935c-d59d278761fc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.631979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.632603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.632948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.633226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.633906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:16.634215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.635638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8c3e-0679-406e-943b-f5ad429cb202 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:16.647126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 175/349] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:16.651275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed31f-cf45-43c2-8bd0-6b13b962a155 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.651275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed31f-cf45-43c2-8bd0-6b13b962a155 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.651275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed31f-cf45-43c2-8bd0-6b13b962a155 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.651275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 176/350] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:16.651275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed31f-cf45-43c2-8bd0-6b13b962a155 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.699134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.699900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.700101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.700386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.701647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.703095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:16.709879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 175/351] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:16.709879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:16.754571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.797720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed31f-cf45-43c2-8bd0-6b13b962a155 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.799519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:16.838580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.839201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.839542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.839843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.845614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b44c0-193c-4147-bd0a-fdfece9d7430 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:16.893438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:16.893978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:16.895289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:16.895289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:16.946665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.948741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7eb22-c745-4213-8d63-3ab4027c47a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:16.979941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:16.981419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.021444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.068474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 177/352] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:17.070143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.070632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.071149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.079197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.097258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:17.097258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:17.107322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 176/353] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:17.108564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.174805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 178/354] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:17.175676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.233803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42def-eb0e-4b01-9ad6-15c83b5eb7e7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.275092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.281656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.282526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.282996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.283445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.291387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.293200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.295615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 177/355] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.296595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.298129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052dde8e-87dd-428b-aff3-24411a107470 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.334755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.340204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.341254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.342820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.342820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.355468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:17.361299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:17.362371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:17.363207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 179/356] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:17.368880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.436109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183b77-7a4f-499f-8d83-4113a5eb412f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.479279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.480193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.480474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.480753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.482328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.484285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0311-ecec-4e2c-aed0-a22c6ca4aeea None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.487011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 178/357] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.488655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef145a-58d5-40f5-aaa9-696c9b5e344e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.488942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef145a-58d5-40f5-aaa9-696c9b5e344e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.489220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef145a-58d5-40f5-aaa9-696c9b5e344e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.490583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 179/358] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:17.491571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c94d-7c29-4f4f-9b61-38c7e78c8c70 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.495769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef145a-58d5-40f5-aaa9-696c9b5e344e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.536197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.536833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.537114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.537384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.538695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.541188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.542845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 180/359] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.545581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.577884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.622311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.635912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef145a-58d5-40f5-aaa9-696c9b5e344e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.679266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246bf72-b125-4a16-b583-694289ce37bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.679821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.680430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.680874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.681149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.682711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:17.683084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.684612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.690558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 180/360] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.692051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.734036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.734657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.734940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.735215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.736184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:17.736493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.737928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.748244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 181/361] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.749436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.859923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94f9f5-0940-463b-92ea-a385364ebb54 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.890807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed72cb-06a3-42b3-b049-de70f01bd5e1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:17.895773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.913532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.914172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.914454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.914728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.919029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:17.933387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:17.937293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:17.938162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.938635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-dbfc756e-4bd0-40aa-8ccf-29e6ab98ea04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:17.939005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:17.940069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:17.940924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:17.941590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.941590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:17.941718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 181/362] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:17.941718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:17.941844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:17.942377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:17.942377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:17.947618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 182/363] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:17.947618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:17.967845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:17.969444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:18.066798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 182/364] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:18.068067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.083578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74017-b08c-40a1-83ec-2a9b2ab92d0e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.134578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.135725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.136213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.136706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.138653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.141149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.143508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 183/365] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:17 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.145151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.220214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc29b8-930d-4133-a14f-4b3df4334f43 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.235616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.244364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.264669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.265654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.266128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.267672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] PATH_INFO: `/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.269801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] RBAC: Authorizing `identity:get_user(user_id=2b58f8743e064fed8dbf6d21eea6f4f9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.271593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.279257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f266e-fe79-478e-8c29-881214fc22ff None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.279844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 183/366] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/users/2b58f8743e064fed8dbf6d21eea6f4f9 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.280980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.322776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.323414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.323699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.323973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.324899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:18.325211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.327516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.337171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 184/367] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.337831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.416791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd162-76dd-428e-85a1-03cab953a7f7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.466680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.467186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.468124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.468124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.468684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:18.469164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.471303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.474547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33562c3d-ca48-40d2-b298-b6a06c15625f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.478274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 184/368] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=1927f695a15842babcb8a7b304466dbb => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.479119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.497124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.522564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.523188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.523514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.523829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.524531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=a95448be579f4092840ecbee1e109310) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:18.524840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.526355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.533787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 185/369] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.534384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.554995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.615471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53f48-e339-4a41-b8ab-f2c0c245bdfa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.659036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.659834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.660201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.660551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.661384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] RBAC: Adding query filter params (role.id=396ddbab5a1f4d4695997f3eb5f4e009, user.id=2b58f8743e064fed8dbf6d21eea6f4f9, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:18.662228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.668415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.683248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 185/370] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/role_assignments?user.id=2b58f8743e064fed8dbf6d21eea6f4f9&scope.system=all&role.id=396ddbab5a1f4d4695997f3eb5f4e009 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:18.683248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:18.710980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46edb9-2cea-4052-8089-0cfbdb83a838 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.772302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.773234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.773691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.774141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.888538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2637-2cab-474a-91c6-b52a8ac48d39 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:18.905348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.907944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:18.938680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:18.940294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:18.966324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:18.966976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:18.967285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:18.967570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:18.991028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:18.996691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:18.997617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:18.998406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 186/371] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:19.004311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.123414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f095-6811-428f-b0d2-c9b9f301ea6f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.135523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 186/372] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:19.142952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.164824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.165434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.165716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.165989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.166659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:19.166962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.170948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.189184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 187/373] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:19.193776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.245480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:19.289730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:19.313805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac4c72-d14c-4a69-9c7b-99c2d6cedf06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.358317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6520-3534-443a-a35d-21700eca0d77 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.360697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.361667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.361948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.362220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.364454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.365926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.368044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 187/374] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:19.373387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.412551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.413692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.414990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.415525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.507319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861962c6-3847-4b12-b16c-01cf3376ff05 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.512030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.513616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.546621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.547483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:19.547681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.548062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.548438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.549687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.552369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.554997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 188/375] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:19.560729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.599477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:19.686412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 188/376] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:19.692210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:19.735049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6674d0-3031-4626-9d1b-7d5f8171c980 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.774882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.775537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.775818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.776126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.776825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=a95448be579f4092840ecbee1e109310) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:19.777136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.778705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.786975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 189/377] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=1927f695a15842babcb8a7b304466dbb => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:19.878895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db7846-a3f0-452b-9d82-beb3f0396f0f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:19.913208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:19.924218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:19.924878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:19.925158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:19.925962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] PATH_INFO: `/v3/projects/82db1e68be334ddf84a8e69d95cda33e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:19.928020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] RBAC: Authorizing `identity:get_project(project_id=82db1e68be334ddf84a8e69d95cda33e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:19.929496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:19.931171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 189/378] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/projects/82db1e68be334ddf84a8e69d95cda33e => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:19.937249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:20.057751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:20.079988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e523-7eaf-4dce-9a19-99d6bf051b6d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:20.126422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:20.127472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:20.127964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:20.128442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] PATH_INFO: `/v3/groups/7076be090047452e92a93293b25037c1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:20.137624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] RBAC: Authorizing `identity:get_group(group_id=7076be090047452e92a93293b25037c1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:20.139820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922277e-ae13-4e56-9a36-3025ecfbc6c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:20.141716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 190/379] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:19 2022] GET /identity/v3/groups/7076be090047452e92a93293b25037c1 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:20.147708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:20.169955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0835-4d3b-4e0a-864c-36d4d74ec37c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:20.170244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0835-4d3b-4e0a-864c-36d4d74ec37c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:20.170530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0835-4d3b-4e0a-864c-36d4d74ec37c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:20.172164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 191/380] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:20.177449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:20.177735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:20.178010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:20.198963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:20.199311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:20.303206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 192/381] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:20.314189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338347d-233d-47d3-8afe-2dbf0cfa1929 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:20.367515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:20.368129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:20.368411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:20.368686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:20.369430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] RBAC: Adding query filter params (group.id=7076be090047452e92a93293b25037c1, role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=82db1e68be334ddf84a8e69d95cda33e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:20.369743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:20.371434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:20.378065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 190/382] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:20 2022] GET /identity/v3/role_assignments?group.id=7076be090047452e92a93293b25037c1&scope.project.id=82db1e68be334ddf84a8e69d95cda33e&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:20.699544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:20.783318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:20.805531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47a47f-c1a7-4923-a297-997052686775 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:20.841933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:20.842563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:20.842839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:20.843108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:20.855445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:20.861122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:20.861997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:20.862785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 193/383] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:20.874817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:20.960113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:21.018147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745d669-1e05-48e1-a86c-8ec8680ceead None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:21.064061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:21.064677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.064961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.065238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.065908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:21.066736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:21.068141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:21.077548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 191/384] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:21.084598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:21.230122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c7046-b1b2-4832-b951-448dfb54174f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:21.284134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:21.284799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.285112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.285390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.291891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:21.293373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8fdaa-9d91-4f90-a919-bb3088e680e5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:21.294955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 194/385] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:21 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:21.301113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:21.370142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f47e-16f3-4d08-b6b2-133c83849e2d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.370464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f47e-16f3-4d08-b6b2-133c83849e2d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.370705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f47e-16f3-4d08-b6b2-133c83849e2d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.372082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 195/386] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:21.379099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a66e-3ade-4a41-8805-4371f42dc80f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.379650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a66e-3ade-4a41-8805-4371f42dc80f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.379941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a66e-3ade-4a41-8805-4371f42dc80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.395597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:21.406348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-91b5a66e-3ade-4a41-8805-4371f42dc80f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:21.407584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91b5a66e-3ade-4a41-8805-4371f42dc80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:21.465339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17dc8-b078-4e15-b52c-c25e2d4a7331 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:21.507786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:21.523128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:21.523853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.524160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.524965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.526394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:21.528575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefb6c1-4430-4115-a93e-c779c613586c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:21.531396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 192/387] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:21 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:21.532744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088de86-0bdd-439c-9132-245340cefc50 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.533032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088de86-0bdd-439c-9132-245340cefc50 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.533308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088de86-0bdd-439c-9132-245340cefc50 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.534945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 193/388] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:21.537895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088de86-0bdd-439c-9132-245340cefc50 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:21.553680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 196/389] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:21.554964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.555276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.555552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.575197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:21.576454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:21.669471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 197/390] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:21.745918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088de86-0bdd-439c-9132-245340cefc50 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:21.797209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:21.797802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:21.798086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:21.798366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/groups/6871cf93e0164455b4849c296e1f41fd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:21.804514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, group_id=6871cf93e0164455b4849c296e1f41fd, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:21.805961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:21.823139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 194/391] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 30 16:04:21 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/groups/6871cf93e0164455b4849c296e1f41fd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:21.896367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:21.993357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:21.993569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.045330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922d188-22ba-4c2e-ac7b-e52dd451dd8e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.085717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.086685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.087168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.087692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.088668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.102832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7a7e1e46-e99c-4c54-80f8-54f4d055f4a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:22.108418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:22.109445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:22.110280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:22.114251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 198/392] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:22.115772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8066-d259-493a-a5fc-5bcb8d4e7277 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.116055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8066-d259-493a-a5fc-5bcb8d4e7277 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.116861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8066-d259-493a-a5fc-5bcb8d4e7277 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.118309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 199/393] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:22.123364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.123645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.123920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.142863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:22.144641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:22.149030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc351-90af-446c-984c-d70e938f2a4e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.186691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.187362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.187649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.187929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.198866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:22.204300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:22.205227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:22.206006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 195/394] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:22.207208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.237120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 200/395] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:22.238701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.301836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.342780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7d6cb-4b5c-4c9d-b460-4efe6cf8348d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.358565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.361816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a245d-f770-482b-ae7c-47c0c8d723eb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.385278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.386542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.387036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.387555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.388454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:22.389096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.391075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.401970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 196/396] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:22.407900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.436629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.437257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.437543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.437818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.438506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:22.438871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.440368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.458484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 201/397] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:22.464159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.558940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa845a-bebb-47e7-b120-90f7b33d1107 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.614640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.615702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.616186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.616785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.618268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.619966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.622052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 197/398] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:22.628193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.661431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d2702-6b97-4681-ba7a-fa8d8604fa77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.681759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.693119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.712844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.713458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.713808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.714096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.715350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.716728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.718682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 202/399] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:22.720061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.757886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5fae5-e8cf-4d7d-abf2-e69013826f63 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.798502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.799143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.799458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.799769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.801407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.802876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.804770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 198/400] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:22.805341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.835332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a360-0d1b-49d3-9a28-0fc019ba0e29 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.872122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.872677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.872964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.873243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.885187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5e8a6232-0531-4a6e-9188-5c9136ebc124 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:22.890774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:22.891907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:22.892868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 203/401] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:22.893644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9f2c-4f4b-4584-837c-37f3c6bb957f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.893644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9f2c-4f4b-4584-837c-37f3c6bb957f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.893922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf9f2c-4f4b-4584-837c-37f3c6bb957f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.899392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 204/402] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:22.900165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf9f2c-4f4b-4584-837c-37f3c6bb957f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:22.930280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911691-499f-4158-a7f5-d1efcea7f6b6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:22.941236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.958211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:22.967848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:22.968610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:22.968930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:22.969218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:22.970993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:22.973500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:22.976208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 199/403] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:22.977068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.030109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf9f2c-4f4b-4584-837c-37f3c6bb957f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.067518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.068095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.068373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.068653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/groups/6871cf93e0164455b4849c296e1f41fd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.070583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, group_id=6871cf93e0164455b4849c296e1f41fd, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.072176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25753b-e0d3-46ea-bdd7-2452739ea8f7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.086008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03dd5f-4091-486a-a48e-012e461ec362 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.099400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 205/404] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 30 16:04:22 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/groups/6871cf93e0164455b4849c296e1f41fd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:23.099400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.099400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.099400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.122317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:23.123573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:23.127821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.128957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.129243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.129512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.130189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:23.130490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.131906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.141382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 200/405] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:23.143907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.197971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:23.243152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 206/406] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:23.244792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.279131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69657364-83cf-4c3d-8d49-5acaff994aa6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.292613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:23.326985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.328047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.328611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.329193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/groups/6871cf93e0164455b4849c296e1f41fd/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.331925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebc43f559f4f08bb972f335ab1837e, group_id=6871cf93e0164455b4849c296e1f41fd, role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.333674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.351740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 201/407] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 30 16:04:23 2022] PUT /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e/groups/6871cf93e0164455b4849c296e1f41fd/roles/1927f695a15842babcb8a7b304466dbb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:23.377478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc27f28-4aad-44a3-b9c6-706199715a9d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.416662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.417415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.417706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.418016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.419175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.420741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.422351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 207/408] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:23 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:23.431634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.577951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:23.578587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.583418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011503c3-0254-4e1e-ac19-4b6e53275bca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.634631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.635703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.636180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.636666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.638112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.639823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.643451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 202/409] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:23 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:23.649254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.745606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33795268-0a55-4b83-9a4e-f08b854e1252 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.753889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:23.820354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.821284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.821766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.822204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.840901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f912-5bf0-4f80-8e1c-065d25a13ac7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:23.843046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:23.853972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:23.854907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:23.855731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:23.858251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 208/410] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:23.866072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:23.923618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:23.924553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:23.924995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:23.925539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310/groups/6871cf93e0164455b4849c296e1f41fd/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:23.930575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] RBAC: Authorizing `identity:create_grant(project_id=a95448be579f4092840ecbee1e109310, group_id=6871cf93e0164455b4849c296e1f41fd, role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:23.932241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140b27e-911e-431c-ae11-d9271ee4116d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:23.961522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 203/411] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 30 16:04:23 2022] PUT /identity/v3/projects/a95448be579f4092840ecbee1e109310/groups/6871cf93e0164455b4849c296e1f41fd/roles/1927f695a15842babcb8a7b304466dbb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:24.041965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55358a1-6d21-4b67-9526-9eed1afee5f8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.082835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5908c-44b7-4abc-adec-cc09337d769e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.083383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5908c-44b7-4abc-adec-cc09337d769e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.083976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5908c-44b7-4abc-adec-cc09337d769e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.085781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 204/412] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:24.091740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.092594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.093035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.093447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.094286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:24.094752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:24.096363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:24.102672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.103523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.103937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.119476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 209/413] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:24.123007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:24.144533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:24.148030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:24.165115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:24.176010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:24.236103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 205/414] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 16:04:24.251347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd0d97-f090-476a-b555-f620fb29def3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.294286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.294947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.295251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.295535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] PATH_INFO: `/v3/groups/7076be090047452e92a93293b25037c1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.296690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] RBAC: Authorizing `identity:get_group(group_id=7076be090047452e92a93293b25037c1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:24.298135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:24.299902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 210/415] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:24 2022] GET /identity/v3/groups/7076be090047452e92a93293b25037c1 => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:24.306103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:24.442863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa7e1d-ca2a-49ef-9859-0d9315cfce65 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.492839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.493489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.493774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.494052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] PATH_INFO: `/v3/projects/82db1e68be334ddf84a8e69d95cda33e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.495200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] RBAC: Authorizing `identity:get_project(project_id=82db1e68be334ddf84a8e69d95cda33e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:24.496680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:24.498727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 206/416] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:24 2022] GET /identity/v3/projects/82db1e68be334ddf84a8e69d95cda33e => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:24.504308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:24.570400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:24.626750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:24.626940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:24.627347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234204-1ae5-44f6-9cc7-d689f8c96763 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.701798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.702384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.702669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.702949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] PATH_INFO: `/v3/projects/82db1e68be334ddf84a8e69d95cda33e/groups/7076be090047452e92a93293b25037c1/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.704910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] RBAC: Authorizing `identity:create_grant(project_id=82db1e68be334ddf84a8e69d95cda33e, group_id=7076be090047452e92a93293b25037c1, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:24.706444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:24.722988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 211/417] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 30 16:04:24 2022] PUT /identity/v3/projects/82db1e68be334ddf84a8e69d95cda33e/groups/7076be090047452e92a93293b25037c1/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:24.783625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dbeb0-9402-48dd-8f57-c4bb60d1e692 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:24.841591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:24.842391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:24.842785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:24.843165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:24.854686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:24.867386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:24.868487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:24.869438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:24.871300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 207/418] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:24.877421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:25.014509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44575d-8394-43e0-b6d2-de012468a8ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:25.050883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:25.051535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.051818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.052089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.052763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:25.053069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:25.054464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:25.063811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 212/419] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:25.069420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:25.163547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:25.179889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecddb-00f2-48d5-8d31-ca2a12026869 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:25.216926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:25.217543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.217826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.218105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.291953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:25.293333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e73fd-ade7-4894-b706-758c60cfd057 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:25.384546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:25.401530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 208/420] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:25.407601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:25.433832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:25.543052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40acb-4dfe-47d6-bef0-9f1ce12daf8a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:25.556949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2449fe-7af5-41c4-9435-38da1d59ceac None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.557255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2449fe-7af5-41c4-9435-38da1d59ceac None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.557511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2449fe-7af5-41c4-9435-38da1d59ceac None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.558858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 209/421] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:25.564086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44a68-fbbb-4124-9870-07b4fea5d902 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.564370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44a68-fbbb-4124-9870-07b4fea5d902 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.564647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44a68-fbbb-4124-9870-07b4fea5d902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.590301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:25.591814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.592102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.592383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.593554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:25.595022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:25.596893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 213/422] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:25 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:25.602781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:25.616662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-adf44a68-fbbb-4124-9870-07b4fea5d902 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:25.617874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-adf44a68-fbbb-4124-9870-07b4fea5d902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:25.691567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:25.709556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086455fa-deff-4aac-8e6b-8c2eb614ea63 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:25.746240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:25.746886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.747175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.747483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.748645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:25.750232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:25.751850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 214/423] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:25 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:25.758773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:25.780295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 210/424] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:25.781865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927d6ad-3055-4b14-a526-68f8a8302f70 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.782343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927d6ad-3055-4b14-a526-68f8a8302f70 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.782808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927d6ad-3055-4b14-a526-68f8a8302f70 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.785139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 211/425] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:25.791411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.791700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.791990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.812628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:25.813794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:25.926044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3e72-eae6-4fb5-89e9-abe0647ef12e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:25.947518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 212/426] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:25.985352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:25.986788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:25.987356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:25.988366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:25.989381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:25.989884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:25.991561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:25.998721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 215/427] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:25 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:26.106998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.182903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.252365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c95a0-ecd7-4e4a-9247-f64cfa651913 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:26.290171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:26.291160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.293424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.293915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.309049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5ed8ba0a-8d35-499b-91b3-6ba3ac706ed9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:26.314195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:26.315064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 16:04:26.318655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:26.321122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 213/428] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:26.332426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.355339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ba4f-e646-437a-ad26-e0405f301b36 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.355964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ba4f-e646-437a-ad26-e0405f301b36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.356433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ba4f-e646-437a-ad26-e0405f301b36 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.358006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 214/429] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:26.362157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.367356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.367356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.367356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.387546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:26.388734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:26.471756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5ab2-1995-4495-8f4e-5e268f53f488 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:26.473385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 215/430] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:26.474029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.523666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:26.524279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.524479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.524753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.525427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:26.525732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:26.527306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:26.536541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.537064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 216/431] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:26.545336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.624995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390f91-1b21-48f0-b43b-500f819789ca None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:26.660362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.669483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:26.670124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.670410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.670692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.685405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:26.686913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a742b45-0f42-439e-92c9-5f438dcf863e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:26.692000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:26.692883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:26.693700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:26.698206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 216/432] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:26.699880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.732095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:26.733200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.733899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.734394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.815535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:26.817657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:26.822445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.829422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59fd4-a9d3-4f3d-aeec-c8b411563a41 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:26.866778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:26.867419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:26.867697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:26.867973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:26.868631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:26.868932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:26.870295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:26.879011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 217/433] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:26.879785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:26.918779 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:26.945639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 217/434] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:26.946388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.006821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af27a5-07e8-4dcb-aa8d-d7a7249f9ae0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.043664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.044301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.044599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.044850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.049839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f567f9-663c-48ac-9c0d-905cc3ad0767 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.056491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:27.062920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:27.063973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:27.064777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 218/435] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:27.066577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.086057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.093889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.094495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.094874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.095145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.146336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.171009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c973b-511e-4387-aa14-57217f382324 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.172017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.174182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.208011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.208687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.208927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.209209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.210413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.212069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.218526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 219/436] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.218526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.287148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 218/437] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:27.288307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.310386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.325482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56551c8c-2369-4454-946a-9e98d5c0c2b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.360069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.360670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.361302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.361676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.362224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:27.362462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.363909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.367655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.377030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 220/438] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.377030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.394857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57ad1-e32d-4f01-83ed-410a1cf620f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.429016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.429644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.429928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.430204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.431346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.432806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a02db-a691-44ce-8e5b-81aff9f6f31c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.434366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 219/439] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.435490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526dd4-aedb-45ac-9591-2329226e03d1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.435784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526dd4-aedb-45ac-9591-2329226e03d1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.436059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526dd4-aedb-45ac-9591-2329226e03d1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.437245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 220/440] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:27.437945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526dd4-aedb-45ac-9591-2329226e03d1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.477101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83445423-41f9-47ed-8a3f-0bdcee78ad7b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.511812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.512429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.512713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.512989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] PATH_INFO: `/v3/projects/d8ebc43f559f4f08bb972f335ab1837e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.514127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebc43f559f4f08bb972f335ab1837e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.515721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.517325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 221/441] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/projects/d8ebc43f559f4f08bb972f335ab1837e => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.518084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.537150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.549041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526dd4-aedb-45ac-9591-2329226e03d1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.582545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.583139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.583446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.583719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.599024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.621646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85f46b-3c72-4264-b00e-6e2c83946877 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.659156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.660272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.660556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.660654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.660832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.661556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31, scope.project.id=a95448be579f4092840ecbee1e109310) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:27.661863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.662208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.663290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03036-17b7-436f-8210-4e9ac04aef7a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.668953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 222/442] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.670101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.670387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.670663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.690562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:27.691743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:27.767843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 223/443] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:27.768787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.792067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.806524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 221/444] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:27.812498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.876696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:27.916256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201630d7-65f0-48b7-ae4d-875cfc0878e4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.950664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1afa-d24a-4f64-88ee-ac1b824a9817 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:27.953274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.953993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.954277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.954549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.955687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.957189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.958750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 224/445] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:27.964503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:27.991560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:27.992291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:27.992768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:27.993031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] PATH_INFO: `/v3/projects/a95448be579f4092840ecbee1e109310` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:27.994176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] RBAC: Authorizing `identity:get_project(project_id=a95448be579f4092840ecbee1e109310)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:27.995656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:27.997488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 222/446] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/projects/a95448be579f4092840ecbee1e109310 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:28.002654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.055382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.077746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d545-2bfa-4b14-b611-522f4fad8d91 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.112125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036186b-51b0-4c9f-b9bb-0b3e8a6cfb32 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.113784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.114389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.114672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.114952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.115705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=d8ebc43f559f4f08bb972f335ab1837e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:28.116014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.117459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.123425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 225/447] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:27 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=d8ebc43f559f4f08bb972f335ab1837e&role.id=1927f695a15842babcb8a7b304466dbb => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:28.124044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.130072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.155698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.156348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.156629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.156901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] PATH_INFO: `/v3/groups/6871cf93e0164455b4849c296e1f41fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.158105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] RBAC: Authorizing `identity:get_group(group_id=6871cf93e0164455b4849c296e1f41fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.160461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.162613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 223/448] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/groups/6871cf93e0164455b4849c296e1f41fd => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:28.168756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.258701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb47d1a-d6b5-48ab-af20-01fe09312137 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.283724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ce78-a7db-4078-b265-9b02bb79ad4c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.298780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.299767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.300425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.300425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.317607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:28.322737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:28.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:28.325065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 226/449] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:28.330521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.334776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.336113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.336709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.337024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.337302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.338208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] RBAC: Adding query filter params (group.id=6871cf93e0164455b4849c296e1f41fd, role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=a95448be579f4092840ecbee1e109310) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:28.338659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.340087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.352031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 224/450] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/role_assignments?group.id=6871cf93e0164455b4849c296e1f41fd&scope.project.id=a95448be579f4092840ecbee1e109310&role.id=1927f695a15842babcb8a7b304466dbb => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:28.461471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668718-466a-4cab-82dc-15c9a5adf75a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.501556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.502429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.502716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.502997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.504192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:28.504505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.505888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.516425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 227/451] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:28.522818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.573831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.631712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d34f3-d045-49f7-9a03-696a45efeb2f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.676925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.677541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.677945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.678106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.747757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.749230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.795487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.851901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 225/452] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:04:28.857637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:28.920223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:28.959207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b2-4c5a-4f5b-9c46-d687a5c4b3b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:28.992127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:28.992743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:28.993027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:28.993301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] PATH_INFO: `/v3/projects/82db1e68be334ddf84a8e69d95cda33e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:28.994420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] RBAC: Authorizing `identity:get_project(project_id=82db1e68be334ddf84a8e69d95cda33e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:28.995889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:28.997505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 228/453] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:04:28 2022] GET /identity/v3/projects/82db1e68be334ddf84a8e69d95cda33e => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:29.003082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:29.101630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5330d5-f5f4-44a8-8cc3-62f0f06be502 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:29.134912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:29.135562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:29.135841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:29.136117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] PATH_INFO: `/v3/groups/7076be090047452e92a93293b25037c1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:29.137209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] RBAC: Authorizing `identity:get_group(group_id=7076be090047452e92a93293b25037c1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:29.138654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7d6eb-089c-4fef-b41d-029a3844c6a3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:29.140225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 226/454] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 30 16:04:29 2022] GET /identity/v3/groups/7076be090047452e92a93293b25037c1 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:29.146599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:29.242946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7effd-a1ea-4a95-a61c-8ffe4fb5486d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:29.271422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:29.276933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:29.277531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:29.277814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:29.278091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:29.278809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] RBAC: Adding query filter params (group.id=7076be090047452e92a93293b25037c1, role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=82db1e68be334ddf84a8e69d95cda33e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:29.279123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:29.280590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e879d54-6b3f-47ad-b6a3-0c4d187b8789 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:29.286308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 229/455] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 30 16:04:29 2022] GET /identity/v3/role_assignments?group.id=7076be090047452e92a93293b25037c1&scope.project.id=82db1e68be334ddf84a8e69d95cda33e&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:31.612473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c4112-727c-4df3-b68e-3b355c2db771 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.613068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c4112-727c-4df3-b68e-3b355c2db771 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.613068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c4112-727c-4df3-b68e-3b355c2db771 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.614622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 227/456] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:31.621107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe238-c568-4b47-ad84-d7336ae9121e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.621616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe238-c568-4b47-ad84-d7336ae9121e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.621724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe238-c568-4b47-ad84-d7336ae9121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.652626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-85dfe238-c568-4b47-ad84-d7336ae9121e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:31.653845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-85dfe238-c568-4b47-ad84-d7336ae9121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:31.741560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 230/457] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:31.929830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf1d-0d1c-4141-807f-178a62218af7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.930109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf1d-0d1c-4141-807f-178a62218af7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.930534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9faf1d-0d1c-4141-807f-178a62218af7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.932609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52400-f954-4fa9-9b12-f47bc3f6c129 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.932899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52400-f954-4fa9-9b12-f47bc3f6c129 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.933280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52400-f954-4fa9-9b12-f47bc3f6c129 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.933480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 228/458] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:31.934509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 231/459] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:31.938231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.938736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.939266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.939475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:31.939979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:31.940496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:31.960471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:31.962072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:31.966038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:31.966655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:31.976880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.064207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 232/460] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:32.078114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 229/461] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:32.163897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.238555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.288810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cc2ce-0f79-4757-bd8f-1a2bce6f5bd2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.326403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.327043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.327349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.327624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.338369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.339874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:32.355887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:32.356841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:32.357724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:32.357724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.357724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:32.357724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 233/462] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:32.364520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.381317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.485471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a26eb-6f58-459e-874b-155201415e88 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.507694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.531031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.531759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.532037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.532306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.539257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f0558-2ce7-48cb-812f-b0124a302d2b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.544954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.546687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:32.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:32.564365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:32.565280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:32.565280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.565280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:32.565280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 234/463] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:32.565280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.583115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.583851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.584200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.584542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.585203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:32.585622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.587141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a8fa-a2ac-40ed-a274-fef64d84e51a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.600029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 230/464] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:32.601078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def888e-3c76-4e0c-ad40-e7bb6d3d120d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.601355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def888e-3c76-4e0c-ad40-e7bb6d3d120d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.601633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def888e-3c76-4e0c-ad40-e7bb6d3d120d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.602959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 231/465] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:32.603548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def888e-3c76-4e0c-ad40-e7bb6d3d120d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.607473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.690780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71e179-f4a3-4ab7-8fd3-7b4e1459e69a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.722272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def888e-3c76-4e0c-ad40-e7bb6d3d120d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.729057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.729627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.730328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.730611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.745058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.746556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.760702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.761229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.761510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.761784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:32.762039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:32.762867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:32.763762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:32.763762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:32.763762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:32.763933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:32.763933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.764448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057e8c-e8cb-4ae1-bc8a-5a43ecc8fcde None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.767970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 235/466] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:32.769245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.782879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 232/467] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:32.782879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dde813-e147-466b-b760-4c62992246ab None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.782879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dde813-e147-466b-b760-4c62992246ab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.782879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dde813-e147-466b-b760-4c62992246ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.790369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.798109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d5dde813-e147-466b-b760-4c62992246ab None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:32.799324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d5dde813-e147-466b-b760-4c62992246ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:32.861600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:32.871778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 233/468] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:32.872939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dde813-e147-466b-b760-4c62992246ab None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.897367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16910e6c-76b3-4695-afab-7f620151954f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:32.947806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:32.949499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:32.949999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:32.950650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:32.951918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:32.953280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:32.983669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 236/469] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:32 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:32.985155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:32.995765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dde813-e147-466b-b760-4c62992246ab None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.038821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.039776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.040198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.040627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.041447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:33.041922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.043520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.057402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 234/470] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:33.068569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.078855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.123246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9550d2-9727-457f-b33d-3cb86fac4460 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.162964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.164163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.164523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.164873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.165644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.167468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.171326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.204629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 237/471] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:32 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:33.204629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.246733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd4a-6ee4-499f-b3fc-670693ef3542 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.295311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.295988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.296277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.296551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.297269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.298757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee060cd-7c5b-4a06-9ac5-ae45c5b2138d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.329159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 235/472] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:33.355350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b768-8785-472f-af64-c692168dc4c9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.357998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fe165-e004-4454-a4b5-b98184edc8c7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.358289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fe165-e004-4454-a4b5-b98184edc8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.358570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fe165-e004-4454-a4b5-b98184edc8c7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.360331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 236/473] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:33.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.369365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.369610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.390550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.400350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:33.401900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:33.406835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.407527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.407821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.408080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.426063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.428029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:33.447935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:33.448916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:33.449743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:33.449743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:33.449743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:33.449743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 238/474] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:33.454998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.507666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 237/475] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:33.579070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f23ef-92a7-4c9b-8bb1-bfcb41a6e278 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.634684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.635778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.636269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.636790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.637644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:33.638171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.639894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.661847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 239/476] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:33.667206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.714593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:33.786896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5797aea-15d1-4b9b-b1f3-94b9fb7edf82 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:33.825799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:33.826422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:33.826706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:33.826983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:33.827669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:33.829166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:33.850716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:33.864649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 238/477] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:04:33.908892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.039984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f9e2-4dbd-4d48-8418-f2f11ad08bbc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.079832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.080357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.080643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.080918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.081671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.083110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.091618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.sql.core [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 16:04:34.091618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 16:04:34.091618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [parameters: {'id': 'a7e2ae8151a14a0bb04d7d2abd9c93bd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 16:04:34.091618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81606) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 30 16:04:34.109847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:04:34.110758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application yield resource Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise self.value Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application session.commit() Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 16:04:34.113025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.flush() Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application compat.raise_( Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 16:04:34.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:04:34.116849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:04:34.125206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [parameters: {'id': 'a7e2ae8151a14a0bb04d7d2abd9c93bd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:34.126298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 16:04:34.127337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:34.131449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 240/478] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 30 16:04:33 2022] POST /identity/v3/roles => generated 144 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:34.131516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.242075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.248893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323782d-f7b0-4da4-a9e9-bd65c43eb61e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.293954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.294654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.294959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.295251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.306340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cddb37a7-1685-4bb7-8c76-0783090c7416 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:34.311798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:34.312671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:34.313875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 239/479] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:34.319623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:34.426597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:34.430734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74398193-8e5e-4258-80c1-2cb823f8006c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:34.468434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:34.469106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:34.469390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:34.469667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:34.470395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:34.470705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:34.472625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39820f-9c8e-4663-9486-d01aaf187d3b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:34.481688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 241/480] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:35.468959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9512-4c01-4d08-9fb7-a0144870db1b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.469994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9512-4c01-4d08-9fb7-a0144870db1b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.470470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9512-4c01-4d08-9fb7-a0144870db1b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.472495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 240/481] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:35.478449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d0ef-8bb7-482a-9324-00613699a468 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.479429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d0ef-8bb7-482a-9324-00613699a468 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.479903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430d0ef-8bb7-482a-9324-00613699a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.512269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b430d0ef-8bb7-482a-9324-00613699a468 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:35.513825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b430d0ef-8bb7-482a-9324-00613699a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:35.586152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 242/482] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:35.806264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d9517-e2eb-41f6-a132-c42a94d555ca None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.806933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d9517-e2eb-41f6-a132-c42a94d555ca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.807429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d9517-e2eb-41f6-a132-c42a94d555ca None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.808948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 241/483] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:35.814108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.814634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.815115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.844250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:35.845760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:35.886249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fb5-3cc5-4065-a8ec-f77e8ca77635 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.887018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fb5-3cc5-4065-a8ec-f77e8ca77635 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.887571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fb5-3cc5-4065-a8ec-f77e8ca77635 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.889657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 242/484] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:35.895262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:35.895796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:35.896284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:35.925935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:35.928002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:35.953399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 243/485] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:35.954791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:35.984219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.036634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 243/486] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:36.068245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d0af-d785-49c9-93a2-6e199957b4a7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.104478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.105579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.106083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.106547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.117990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:36.123122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:36.124049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.124837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:36.127702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 244/487] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:36.134323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.267422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.286154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556f7b4-4f2d-421a-ba17-9d38bcd0b44c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.310514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.349411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.349942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.350225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.350500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.351180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:36.351513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:36.352917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:36.372612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 244/488] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:36.377157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.431729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.451626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69c8a8-6be1-4d64-9f84-f78469ee99aa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.492045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.492799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.493008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.493286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.494125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c331-0368-4b15-9a8a-1a6342d242f3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.506592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:36.513919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:36.514792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.515602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:36.516826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 245/489] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:36.517637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.540629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.541594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.542090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.542552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.554752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:36.556671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:36.566458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:36.576037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:36.576948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:36.577791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:36.577791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:36.577791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:36.580043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 245/490] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:36.581039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.654939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15e975-1da8-4249-8322-4c62a55bc85b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.691974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.692656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.692884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.693154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.704195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:36.709599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:36.710530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:36.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:36.715323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 246/491] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:36.715984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ffe0-2d1d-4880-ad57-4b1f472d8d69 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.716180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.720415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.755676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.756687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.757164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.757701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.758562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:36.759079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:36.760699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:36.770159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 246/492] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:36.771533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.838702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddb6dd-4a15-4b1c-9732-949676907203 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.840674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:36.875166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.875798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.876078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.876351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.876999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:36.877359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:36.879327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:36.883461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ca30-fd9b-401d-9565-e84e2388b075 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:36.896714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 247/493] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:36.896714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.922490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:36.923118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.923423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.923703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.924371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:36.924680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:36.926656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c0230-9fca-4049-b055-eeb9dfdd3cdf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 247/494] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aec926-883a-4541-854f-06a6c3213cd6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aec926-883a-4541-854f-06a6c3213cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aec926-883a-4541-854f-06a6c3213cd6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 248/495] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:36.939884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aec926-883a-4541-854f-06a6c3213cd6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:36.969313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.012797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813585-fc17-4822-97bc-c7b7bea11600 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.047344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aec926-883a-4541-854f-06a6c3213cd6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.048879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.049517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.049802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.050610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.061228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.062689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.071885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:37.077520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:37.078450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:37.079767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:37.079767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:37.079767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:37.079767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 248/496] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:37.079767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.087512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.088150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.088487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.088763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.097479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.098935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0fa54291-9cb7-4146-9d82-1103c9f5d868 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:37.111871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:37.113289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:37.114222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 249/497] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:04:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:37.114797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39121f7-2543-44e0-be30-a99fd9adda03 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.114861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39121f7-2543-44e0-be30-a99fd9adda03 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.115122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39121f7-2543-44e0-be30-a99fd9adda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.132661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e39121f7-2543-44e0-be30-a99fd9adda03 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:37.133867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e39121f7-2543-44e0-be30-a99fd9adda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:37.184553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e7a9-20cc-468a-803f-bb00bf074d31 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.197998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.199645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 250/498] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:37.201008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9688c-3c82-4779-85af-df6c0916310a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.201294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9688c-3c82-4779-85af-df6c0916310a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.201568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9688c-3c82-4779-85af-df6c0916310a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.202933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 251/499] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:37.203674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9688c-3c82-4779-85af-df6c0916310a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.228167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.228829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.229109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.229381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.241604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.243057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:37.258109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:37.259040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:37.259937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:37.259937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:37.259937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:37.259937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 249/500] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:37.260151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.327317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9688c-3c82-4779-85af-df6c0916310a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.343524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.367667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b1b5d-1fea-4d03-9a9b-8e63f0b3dbd4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.368156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.368753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.369073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.369300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.369933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:37.370805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.371631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007917a-c6ad-41db-9205-69a84387dd6e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.383368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 252/501] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:37.384567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.384854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.385126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.404477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:37.405650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:37.413426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.414797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.415395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.416335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.417285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:37.418219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.420492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.459967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 250/502] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:37.460739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.482184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 253/503] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:37.483032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.488110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.569434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3be1a2-e9d7-4f85-820b-94d62ecbed36 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.595980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ef97a-63e3-4566-9d1a-cf1592509cb1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.599063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.608529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.609300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.609585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.609862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.610543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:37.610851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.612566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.629749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 251/504] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:37.630844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.644379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.645185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.645657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.646030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.654372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.655856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:37.667751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:37.668644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:37.669535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 254/505] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:37.670090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.736982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.759076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b71517-edef-4738-9b8f-c2288d33e94a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.788084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ce7c-2f82-47a1-b28c-86996aec59ed None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.796929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.797538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.797821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.798100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.826715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.827379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.827656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.827932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.838905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:37.844923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:37.845898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:37.846728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 255/506] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:37.848035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.871481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.872556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.873343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:37.948765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222189c-4495-4a6e-8041-0fd4bc64a5d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:37.959140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:37.976965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 252/507] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:37.977802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:37.983240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:37.983869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:37.984150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:37.984419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:37.992397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:37.993778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:38.004972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:38.006029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 256/508] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:38.007132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.086017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87122-1f92-4328-acaf-e26f55ab0329 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.102824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.119491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67005e74-43e3-4dc1-ac96-99b701a50e90 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.120116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.120721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.121006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.121280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.122017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:38.122325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.123814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.141352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 253/509] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:38.142163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.156160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.156794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.157077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.157422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.168571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.170020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:38.184552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:38.185588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:38.186472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:38.186472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.186472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:38.186659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.186730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 257/510] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:38.186730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.245902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5a2b-90e3-4b99-a9d8-188a9dcad2be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.279134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.279755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.280037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.280306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.280968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:38.281270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.282637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.288183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5384fe-a7b6-4955-af53-6420ccfdf79f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.290988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 254/511] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:38.291808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.322703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.323367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.323650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.323924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.326712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.335733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.337210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:38.351383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:38.352315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:38.353179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:38.353179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.353179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:38.357063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 258/512] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:38.357850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.391521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766529dc-1009-41b8-ab4d-ad6299ac88d9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.393820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.426052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.426648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.426931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.427211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.427955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:38.428255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.429656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.446814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 255/513] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:38.447632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.459819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefb19d-66cf-48d5-bff6-3e85bec60f65 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.495706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.496358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.496584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.496858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.549611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263374a5-7b4d-4724-a9ae-5916203a3f56 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.554410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.568780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.570263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.583886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.584482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.584765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.585032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.585662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:38.585961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.587348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.598335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 256/514] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:38.599347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.626442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.675953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 259/515] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:38.676810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.704336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b2c8-ecce-4fc3-9667-99b8865258da None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.738287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.738911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.739189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.739500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.750356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.751881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.765624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:38.766171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:38.767180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:38.768164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:38.768164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:38.768164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:38.768164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 257/516] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:38.768164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.785276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca668-6044-4f29-bcfb-27b5c94f5e02 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.821121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.821771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.822053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.822320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.822959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:38.823387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.824653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.836724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 260/517] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:38.837513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:38.852946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:38.879680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6176a7-889a-42f0-b2c3-e8d75c686979 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.914671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.915292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.915582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.915858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.939306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693ba71-1d5a-4b64-9c60-23d3d09ddb19 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:38.973909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:38.974512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:38.974793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:38.975067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:38.975748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.977246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.984859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:38.986550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:38.998936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 261/518] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:38 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:38.999793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.009068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.084411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.115210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 258/519] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:39.116521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.134788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fd277-7538-4649-8032-bc19ccde7e96 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.177271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.178206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.178486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.178759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.191042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.192556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:39.207981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:39.208918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:39.209773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:39.209773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:39.209773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:39.209773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 262/520] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:39.210402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.238443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75de7ca-82a0-4028-81ce-47ab6e97f6b7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.271883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.277145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.277908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.278189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.278758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.279698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:39.280005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.281620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.301622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 259/521] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:39.301622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.317722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008023c4-a819-4f45-befc-656c2437d5ca None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.344961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.353033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.353700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.354119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.354285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.361671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.363739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:39.376772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:39.377651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:39.379094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 263/522] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:39.380049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.425555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb8cf5-705e-4ab3-b838-1a07ee6d87dc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.466668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.467346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.467636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.467915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.480493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.481980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.497420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841a29e-fd4b-4fb0-8d1c-4dfdaeca0a8a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:39.499658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:39.500705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:39.501882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:39.501882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:39.501882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:39.505493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 260/523] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:39.506176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.520732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.540746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.541813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.542191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.542580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.543263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:39.543634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.545641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.560407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 264/524] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:39.562222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.604103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.619359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8b46-3605-42a8-ab6f-0f3c924c8740 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.660901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.661635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.661919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.662191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.670425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.672016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.679361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29417e-185f-4b6e-9c8a-22dbfbbc40e7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:39.685533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:39.686410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 261/525] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:39.688018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.724769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.725401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.725682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.725955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.726656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:39.726961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.728431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.748690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 265/526] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:39.748690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.783755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.806440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3891e52-45f0-4b20-bfa3-ce77b9bdd7b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.842750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.843404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.843667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.843940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.844620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:39.844874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.846238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.853636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd42014-2dc5-42f0-b228-5c76ebef798d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:39.864212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 262/527] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:39.864864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.873368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:39.899442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:39.900073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:39.900357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:39.900630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:39.907559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:39.909224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:39.921233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:39.922208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:39.923044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 266/528] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:39.924100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:39.977704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef5de-d2c6-49f2-b89b-d9ddc524f6c6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.015111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.015990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.016278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.016552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.017295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.017601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.019090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.027011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.034363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a4eed-08c7-4e81-9fed-fe923e118409 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.037339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 263/529] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.038155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.075074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.075730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.076074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.076352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.077080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.077390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.078835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.085326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 267/530] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:39 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.086120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.123374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.175254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c651f2-37c2-4cfb-9349-8dc2948764c7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.214906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.215995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.216280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.216620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.226516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.228642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.239477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a2749-c5de-4718-b287-13fbc9340ffa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:40.244759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:40.245795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 264/531] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:40.248029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.282868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.283992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.284697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.285078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.285901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.286569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.288798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.306798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 268/532] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.307986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.333464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.373017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197dd50-a457-49ab-ad46-fac123e9b6e2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.401344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.418682 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.420016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.420409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.420958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.430821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac73c4f-1047-4b01-a65a-51c9d4591e7e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.470787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.471756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.472187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.472687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.473559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.474028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.475668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.489412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 269/533] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.490675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.511647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.513587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.598516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.623910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153c7f5-5a12-450a-86c9-50cf8e544ac9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.646401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 265/534] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:40.647935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.673120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.673881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.675376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.675751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.676769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=de5581201518429191f74cf8af19fb58) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.677239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.679669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.679786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.695259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 270/535] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/role_assignments?user.id=de5581201518429191f74cf8af19fb58&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.695259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:40.851997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3040-17d2-44c2-80f7-8aab8603aaae None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.898455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616652cc-0afa-4f46-94b5-f28630444e5e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:40.938150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.939159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.939778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.940384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.943934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:40.944557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:40.944845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:40.945123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:40.945862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=c34f7dd7c12a4262a6ace2f07b6475c1) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:40.946178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.947640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.954131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:40.957066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:40.960415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:40.964447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 266/536] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/role_assignments?user.id=c34f7dd7c12a4262a6ace2f07b6475c1&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-88f1bb62-1c35-44f0-8d40-9e7bd1fa5723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:40.976181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:40.977154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:40.977998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:40.977998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:40.977998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:40.979976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 271/537] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:40.987928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:41.085774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e4212-a549-4428-ac9a-5ca2ca0f21ec None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.086129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e4212-a549-4428-ac9a-5ca2ca0f21ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.086335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e4212-a549-4428-ac9a-5ca2ca0f21ec None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.090401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 272/538] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:04:41.096966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.097477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.097981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.137702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:41.140588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:41.151052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:41.164944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74810eb7-a783-450d-b35c-b020e82b9065 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:41.206769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:41.207412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.207751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.208034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.208889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:41.209080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:41.210532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:41.223132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 267/539] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:41.229065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:41.280520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 273/540] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:41.355169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7c054-e191-4806-87c6-5c1625faa8b6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:41.391151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:41.391866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.392143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.392413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.400118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:41.401684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:41.414364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:41.415270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:41.416122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 268/541] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:41.421681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:41.545825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70db07-70d8-4b80-8f63-82b7a9ca4b31 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:41.582085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:41.592700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:41.593405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.593691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.593971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.594678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:41.594988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:41.596590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:41.608727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 274/542] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:41.617336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:41.635925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:41.698332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:41.788018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd610f-ad5c-4ad5-bb84-87f06800da1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:41.858565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4ffb7-983d-4329-9522-c077abc5ce10 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:41.878356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:41.878982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.879282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.879560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.880296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=393f32c7fcdf49219d155bb22fd3d048) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:41.880608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:41.882073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:41.899343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 269/543] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:41 2022] GET /identity/v3/role_assignments?user.id=393f32c7fcdf49219d155bb22fd3d048&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:41.909749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:41.910503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:41.910860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:41.911208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:41.923809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:41.930275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:41.931523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:41.932508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:41.934055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 275/544] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:41.939369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.023106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:42.061697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330eab-fa41-473f-8a37-02c306196018 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:42.100110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:42.100726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.101008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.101284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.101969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:42.102272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:42.103703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:42.115628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 270/545] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:42.121368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.174278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:42.238674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe9ce4-c859-4feb-bfac-e6c4d6bb6312 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:42.281292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:42.281930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.282210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.282483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.297695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:42.299155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7340509e-3cc5-404d-b033-73681c09706c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:42.319443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:42.320398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:42.321237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:42.321237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:42.321237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:42.321237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 276/546] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:42.327498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.336784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de736-d218-4f19-8a6a-20a99ec5508c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.337093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de736-d218-4f19-8a6a-20a99ec5508c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.337340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57de736-d218-4f19-8a6a-20a99ec5508c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.338965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 277/547] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:42.348881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.349154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.349431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.380393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:42.381623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:42.457070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4497-9a80-44a6-936c-f0879de52b8f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:42.494817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:42.495458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.495745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.496021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.496667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:42.496972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:42.498340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:42.499870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:42.502242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 278/548] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:04:42.513019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 271/549] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:42.518518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.554519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:42.663319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcb9e0-02fc-4f33-8575-918b49d5dfb5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:42.703365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:42.704113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.704295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.704507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.712627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:42.714010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:42.727458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:42.729971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:42.732264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 279/550] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:42.736592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.854190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c391af-38e5-4d42-be99-c0dd8ee1715d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:42.866167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.890404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:42.891019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:42.891328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:42.891647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:42.892392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:42.892706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:42.894190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:42.912849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 272/551] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:42.918326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:42.990319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.023687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16207c-c579-4fa0-a37e-b78943a26f9e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.040678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.058010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c8a10-cdae-4070-aaea-c45262abe77f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.063448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.064060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.064341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.064617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.103526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.104161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.104444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.104734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.127612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-58298794-07e9-499d-9d2b-429b80be1221 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:43.135270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:43.136450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:43.137248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:43.139295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 280/552] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:43.140832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8af8fa-e4b3-48c7-afa2-5f37aae11ff7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.141356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8af8fa-e4b3-48c7-afa2-5f37aae11ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.141675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8af8fa-e4b3-48c7-afa2-5f37aae11ff7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.143082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 281/553] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:43.145064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8af8fa-e4b3-48c7-afa2-5f37aae11ff7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.152361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.153816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725380-1097-4135-a5eb-09466cbc2edf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.211329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.271335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 273/554] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 16:04:43.271335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.271335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.271335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.282250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8af8fa-e4b3-48c7-afa2-5f37aae11ff7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.288555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:43.289747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:43.328701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.329679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.330149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.330569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.332214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:43.333693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.335305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.347842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 282/555] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:43.347842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.370455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.372800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 274/556] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:43.373683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.470292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.495045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85499f-14f1-4eee-a445-f4161ae901c8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.525728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397fae8-2a97-4ba2-b293-7993278af53d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.542886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.543804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.544476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.545305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.561467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.563374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.563816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.564543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.564948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.565360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.576942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.578524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ae004ba8-c9ff-45a8-b40e-7e7079e1abaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:43.582150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:43.583082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:43.584025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:43.584025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:43.584025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:43.586285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 283/557] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:43.587546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f6ce-2189-4016-bd72-e8266c46dd99 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.587653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f6ce-2189-4016-bd72-e8266c46dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.591568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f6ce-2189-4016-bd72-e8266c46dd99 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.593465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 284/558] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:43.594193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f6ce-2189-4016-bd72-e8266c46dd99 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5fca59de-9158-4565-ac8e-41ea62556d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:43.611754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:43.613403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:43.614285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:43.614285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:43.614285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:43.614285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 275/559] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:43.619184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.620503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.620959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.650730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:43.652040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:43.678009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.722482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f6ce-2189-4016-bd72-e8266c46dd99 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.725426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 276/560] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:43.726317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.778275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.782706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.783243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.784040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.785223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:43.785731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.787942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.792731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.811996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 285/561] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:43.813183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.845092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99924d2e-d951-45a6-bb1b-534c61938d29 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.897272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.899814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.900265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.900924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.902337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:43.903157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:43.907601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:43.922247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 277/562] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:43.923328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:43.939864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c087-a309-46cc-acd8-3a162cf993f2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:43.964388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:43.978528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:43.979495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:43.979967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:43.980415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:43.996450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:44.002056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.003060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:44.003963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 286/563] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.004672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.040368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00616c-37d6-4ecd-a738-0899fb03f71b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.063961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.075953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.076587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.076868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.077149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.084692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.086133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:44.097443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.098344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:44.099239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 278/564] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.099737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.123751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfbfc-a422-4feb-8b76-6949bcdb2992 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.163600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.164225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.164501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.164773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.172737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.174207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:44.192599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.193628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:44.194610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 287/565] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.199467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.221665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78af2d-f6e6-4ab2-8d14-887674ff8330 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.247109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.258382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.259104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.259503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.259854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.271578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:44.276956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.277887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:44.278729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 279/566] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.280226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.320733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6dc39-5529-4b75-880f-a26b3881b9dd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.337056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.358783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.359422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.359707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.359981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.360649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.361217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.362663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.371115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 288/567] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.371948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.404375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e8b5d-d10b-4bfa-9328-3dc7e593ec02 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.438288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.438885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.439175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.439475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.440168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.441100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.442524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.459750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 280/568] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.460564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.475881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664b002-672a-47d4-b13c-f87a77caa819 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.478788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.512647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.513249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.513534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.513810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.514536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.514843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.516315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.552744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 289/569] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.553567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.568052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed0ba-82c9-41a1-96d4-b86fc810c5af None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.569784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.601661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.602184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.602465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.602738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.603431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.603735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.605116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5a69c-c4b8-4e6c-8745-d82ebc8d9415 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.613662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 281/570] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.614813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c7512-d3e7-461a-8eef-eece37a5b02b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.615105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c7512-d3e7-461a-8eef-eece37a5b02b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.615417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c7512-d3e7-461a-8eef-eece37a5b02b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.616879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 282/571] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:44.617586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c7512-d3e7-461a-8eef-eece37a5b02b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.659913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53934655-3c28-49fd-ad5c-b37a86f0e3e6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.693975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.694591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.694866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.695135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.704974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.706477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:44.720814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.721942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:44.728305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:44.728305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.728305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:44.728305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 290/572] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.728305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.728595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c7512-d3e7-461a-8eef-eece37a5b02b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.738957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.761111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.761704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.761986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.762255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.762967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=058f62fc005a41b08ee30fb42192e31a) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:44.763302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.764729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.770523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 283/573] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/role_assignments?user.id=058f62fc005a41b08ee30fb42192e31a&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:44.771353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:44.815877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:44.845450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f8684-870e-472f-b2d2-64016fb9efb7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.886749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.887856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.887856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.888026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.902877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.903091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303fca-4440-4afc-80a2-e81f68dbf897 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:44.904413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4c3b14d9-52de-48fe-8059-3245046bc8fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:44.920958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.921970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:44.922866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:44.922866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.922866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:44.922866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 291/574] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.923310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.923610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.923867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.942996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:44.944301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:44.945294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:44.946024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:44.946375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:44.946720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:44.959042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:44.960528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:44.976125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:44.977117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:44.978024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:44.978024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:44.978024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:44.978024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 284/575] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:44.978024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.014014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.024522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 292/576] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:45.025246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.083306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.107766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6488c13-4753-483f-bda2-2fba3c5ed5d2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.143259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.144167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.144597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.145017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.145881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.146388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.147987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ac1d-2620-4f88-80e0-e8b55825a9d1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.148208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.167614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 285/577] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.168531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.186131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.187107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.187628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.188325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.189198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.189716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.191485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.204426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 293/578] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.205703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.279636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.291515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e77ff-855b-4aca-ad12-c0f28d88ba9b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.328487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.329083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.329363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.329462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879465-4242-4d05-b3f5-bb3451921abb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.329636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.330268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.332492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.344128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 286/579] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.344930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.348044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.373730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.374361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.374639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.374906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.383875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.385331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:45.397687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:45.398647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 294/580] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:45.404527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.458225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eac1-a4d3-4ce9-8b8c-c378061248eb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.494387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.495047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.495361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.495644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.504783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.506246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.515019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14828dd1-c0f6-428f-96c5-2721c9e3740b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:45.518636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:45.519578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 287/581] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:45.525604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.537077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.557617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.558443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.558868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.559335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.571569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:45.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:45.578705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:45.579746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:45.581743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 295/582] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:45.582676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.597664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.657131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e928-ff2d-4a82-b5b3-5b0199ddfeea None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.692118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.692800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.693092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.693366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.701157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.702588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.709967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ab4f8-86cb-477e-b432-de759908c13b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:45.714264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:45.715196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 288/583] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:45.721015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.745543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.746255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.746462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.746693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.747575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.747879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.749311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.771755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 296/584] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.773000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.813550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.836174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747310c2-27f0-4b0a-90f4-d8284ad08d72 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.864450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:45.876426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.877033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.877317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.878308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.878829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.879139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.880594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.891816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f628fd-7b6b-4cd5-8c75-a4a680e0816f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:45.894421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 289/585] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.899912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:45.933067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:45.933570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:45.933853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:45.934123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:45.934793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:45.935096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:45.936535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:45.949378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 297/586] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:45.949378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.009672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e17c91-5209-4243-b455-2c7aac3d1ae9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.046602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.047211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.047515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.047751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.047889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.048866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:46.049233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.051489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.055544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fd96e-9b5d-4bed-a341-674c9ea64a30 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.070869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 290/587] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:46.071838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.098562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.099165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.099474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.099743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.112009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.114373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.121795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:46.128510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:46.129665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:46.130726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:46.130726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.130726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:46.136028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 298/588] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:46.136028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.191239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e37d-218d-4acd-a411-3fb9dc0cea5d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.227762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.228716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.229056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.229277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.247364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=5ce87b2207504e839bcd544faa3aa7ad, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.248815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.251415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fde015-f558-4496-bcbb-612344b1df06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.261532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 291/589] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:46 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:46.262462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.309044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.309969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.310257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.310587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.327361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.329694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.342175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:46.353307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:46.354348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:46.355275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:46.355275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.355275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:46.355275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 299/590] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:46.361786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.392649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.408635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c44a-4ca5-412c-9011-98f6c23bed75 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.456309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.457360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.457869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.458251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.476502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.478844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:46.500902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:46.502054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:46.503752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:46.503752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:46.503752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:46.506489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 292/591] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:46.507468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef0653-6b03-4a12-a388-02070e49fd4e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.507725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.554496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c79a-b924-4802-a106-233302edacac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.555110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c79a-b924-4802-a106-233302edacac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.555440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c79a-b924-4802-a106-233302edacac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.555715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c79a-b924-4802-a106-233302edacac None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.556366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c79a-b924-4802-a106-233302edacac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:46.556671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c79a-b924-4802-a106-233302edacac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.558056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c79a-b924-4802-a106-233302edacac None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.571504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 300/592] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:46.572489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c79a-b924-4802-a106-233302edacac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.626402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777d778-7caa-4b9a-aa1e-2f7bacc27cd7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.635927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.665251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.666139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.666563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.666976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.667852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:46.668312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.669817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.680898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c79a-b924-4802-a106-233302edacac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.683392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 293/593] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:46.684359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.689906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.728535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.728857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.729041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.729316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.729955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:46.730339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.731695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.744114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 301/594] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:46.745086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.817190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2bbf8-b86d-4679-b36b-e696dd8cf56a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.854576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86922ada-0a74-4a38-96df-6abe33ad69dc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:46.855900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.856536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.856822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.857104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.864739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.866181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.876756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-829034e5-9e9a-411d-8ce8-8b841b6b81bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:46.877545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:46.879465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:46.880362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 294/595] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:46.881190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054a77d-19ed-477d-b17d-05efe94728b2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.881480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054a77d-19ed-477d-b17d-05efe94728b2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.881762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054a77d-19ed-477d-b17d-05efe94728b2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.887057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 295/596] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:46.887057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054a77d-19ed-477d-b17d-05efe94728b2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.903730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:46.904359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:46.904665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:46.904940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:46.912975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:46.914448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:46.926135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:46.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 302/597] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:46.928456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:46.963925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.007545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054a77d-19ed-477d-b17d-05efe94728b2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.043596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6145ca-dced-4e6d-839c-65dadf665f3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.045456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.046618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.047044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.047499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.055125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.056811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-50788339-aad7-46ac-8f69-60f2e665a73f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:47.069488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:47.070620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:47.071645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:47.077947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 296/598] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:47.079283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.079626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.079853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.081194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.081829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.082106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.082374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.083068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:47.083404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.084838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.103989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 303/599] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:47.103989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.109502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:47.110720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:47.156524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.199271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 297/600] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:47.199271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.229807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530b3dd-8478-4654-b9b2-3e1c28204344 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.263670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.276210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.276830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.277116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.277390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.278385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:47.278706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.281093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.305943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 304/601] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:47.306899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.319740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2506-709d-42ad-b82b-b0891a312221 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.356102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.357076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.357569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.358036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.358935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:47.359510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.361377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.373912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 298/602] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:47.375055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.430416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.436158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911d28-cabb-4205-aeb2-817a025ad35f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.479717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.480495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.480808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.481169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/de5581201518429191f74cf8af19fb58/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.487754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf23ba-eda3-45c2-a8ca-91715e0059d3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.497626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=de5581201518429191f74cf8af19fb58, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.499094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.500951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.519314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 305/603] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:47 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/de5581201518429191f74cf8af19fb58/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:47.519314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.539321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.540283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.540572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.540849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.558182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.559713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:47.581856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:47.582874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:47.583778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:47.583778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:47.583778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:47.587396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 299/604] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:47.588431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.655665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07f80b-3d2f-4e7b-b730-0bedd5ee5b99 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.695739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.696815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.697176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.697526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/c34f7dd7c12a4262a6ace2f07b6475c1/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.715512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=c34f7dd7c12a4262a6ace2f07b6475c1, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.717370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.732738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.733532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342916-7bc9-4fa3-899a-c0e8643f5407 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.739870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 306/605] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:47 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/c34f7dd7c12a4262a6ace2f07b6475c1/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:47.740731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.810090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.810832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.811285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.811575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.839972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:47.845589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:47.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:47.847938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:47.855778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 300/606] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:47.864353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:47.870488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0294d-0d53-4746-afe3-262a1b437394 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:47.916174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:47.916930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:47.917248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:47.917489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:47.918424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:47.918729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:47.920174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:47.933927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:47.942302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:47.944444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.053881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:48.077770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ecde-dbee-4cb0-b8f9-932b060586e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.104904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4db57-15c2-4cc1-9dd4-5d997597b250 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.116908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.117525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.117878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.118160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.118829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:48.119135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.120737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.130800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 301/608] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:48.136438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.144320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.144956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.145241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.145518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.152904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.154442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:48.184408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:48.185433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:48.186300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 308/609] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:48.191668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.233620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:48.250451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347c150-dd70-4d55-8346-b098985a0e54 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.291625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.292589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.292879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.293158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.306642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.308747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-22678db0-4f94-4655-8804-f202a6937301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:48.327099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:48.328155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:48.329053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:48.329053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:48.329053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:48.329053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 302/610] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:48.330445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22afc-c8e4-42ed-89cb-120694aa800b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.331255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22afc-c8e4-42ed-89cb-120694aa800b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.331603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf22afc-c8e4-42ed-89cb-120694aa800b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.333136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 303/611] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:48.336108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf22afc-c8e4-42ed-89cb-120694aa800b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.360274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b542-8895-47ac-b672-5844743484ef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.379890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:48.399480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.400095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.400381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.400658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.401470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:48.401944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.403409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43bab7-7d1b-41c2-891f-7b31556961eb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.416818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 309/612] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:48.418106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.418424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.418776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.438005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:48.439458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:48.446313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf22afc-c8e4-42ed-89cb-120694aa800b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.482707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.483348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.483633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.484093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.485058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:48.485367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.487609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.497058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:48.508441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 304/613] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:48.509347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.523715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 310/614] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:48.529934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.713510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4ee81-f291-4ce9-93b1-4fc77ce7df30 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.727721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20d832-74cc-46f2-92fa-0414cfe2e44b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:48.747166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:48.792667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.793369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.793651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.793930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/393f32c7fcdf49219d155bb22fd3d048/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.813936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:48.814584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:48.814919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:48.815797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:48.821205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=393f32c7fcdf49219d155bb22fd3d048, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.822917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.823668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:48.825182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:48.838709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:48.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:48.840774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 311/615] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:48.841371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.843178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 305/616] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:48 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/393f32c7fcdf49219d155bb22fd3d048/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:48.845367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:48.907650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.012394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d4e9-d94d-4fb9-be98-dff478d5f736 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.055966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad132-4cbe-4564-8ee5-540b74fda0ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.060158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.060158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.060378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.060610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.061250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:49.061554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.062978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.086569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 306/617] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:49.091948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.101945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.102580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.102857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.103124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.117994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:49.123497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:49.124484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:49.125354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:49.131102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 312/618] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:49.131885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.172073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.264219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a9071-e9c6-4d95-be44-3c8800aa7245 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.305782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a17d32-6e6e-454d-99dc-55e780a64227 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.321018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.332582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.333482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.333776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.334029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.334700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:49.335000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.336398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.357176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 313/619] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:49.363610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.383474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.384111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.384385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.384644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.407719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.410146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:49.442306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:49.443492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:49.444403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:49.444403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:49.444403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:49.447332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 307/620] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:49.449242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.532259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.549754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ef116-c355-4204-849e-8c58ad3ca65e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.591322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.591952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.592229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.592500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.604850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.606309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:49.632510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:49.633509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:49.634411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:49.634411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:49.634411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:49.638479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 314/621] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:49.639849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.646646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aa198-e5e1-4a8b-be4a-3d18d4ca5a6a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.709189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.709715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.710001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.710270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.710914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:49.711238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.712603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29344508-e205-4c8e-b7f7-138662ad03a0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.730285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 308/622] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:49.731666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683437dd-8a75-487a-bb53-c0eb109eb0d6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.731951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683437dd-8a75-487a-bb53-c0eb109eb0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.732226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683437dd-8a75-487a-bb53-c0eb109eb0d6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.734186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 309/623] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:49.734926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683437dd-8a75-487a-bb53-c0eb109eb0d6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.757558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.792296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47262d-b382-4329-8c88-c457e0e76e87 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.833809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.834418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.834697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.834970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.835651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:49.835959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.837514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed2f56-11c0-4c79-93e8-c949de2e5cd4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.846658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683437dd-8a75-487a-bb53-c0eb109eb0d6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:49.850964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 315/624] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:49.852233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.852516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.852857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.859041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:49.874311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:49.875800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:49.889513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:49.890175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:49.890449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:49.891096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:49.899021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:49.900539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:49.913504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:49.914590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:49.915678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:49.919725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 310/625] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:49.920554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:49.958800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 316/626] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:49.960049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.004524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.028871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32a28-aed0-4bbf-a238-086a373a39ed None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.064503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.065981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.066273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.066549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.071491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceed7d-90e6-4fc3-899d-604229e7d956 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.076181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.077636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:50.090101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:50.091508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:50.092448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:50.096003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 311/627] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:50.098057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.135888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.146001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.147032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.148929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.149983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.150890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:50.151583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.155085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.177019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 317/628] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:50.183652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.254006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515fb70-725f-4e6f-9e5d-22320d1f6a1c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.297359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38983f52-c3e6-4c90-b46c-d5bdff4d68fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.301005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.301845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.302323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.302790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.303741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:50.304279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.306431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf7ace-4251-4732-bffa-0d44ca7394c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.318675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.332930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 312/629] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:50.334568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91505b61-4645-49ad-b5a2-a8dc657975ef None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.334855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91505b61-4645-49ad-b5a2-a8dc657975ef None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.335129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91505b61-4645-49ad-b5a2-a8dc657975ef None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.337476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 313/630] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:50.338166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91505b61-4645-49ad-b5a2-a8dc657975ef None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.345667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.346962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.347137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.347438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/058f62fc005a41b08ee30fb42192e31a/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.363286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=058f62fc005a41b08ee30fb42192e31a, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.365168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.378146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 318/631] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:04:50 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/058f62fc005a41b08ee30fb42192e31a/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:04:50.388263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.454360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.467973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91505b61-4645-49ad-b5a2-a8dc657975ef None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.517383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.518104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.518999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.519466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.528107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b843-ea12-47e5-9fc3-77dd66d88f77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.538533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8e9df61a-2ea7-49b3-88a5-0c1dcca603b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:50.544141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:50.545058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:50.545904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:50.547934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 314/632] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:50.549263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.550281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.550690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.569081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.569781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.570065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.570334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.570608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:50.572033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:50.639096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.662949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 315/633] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:50.663913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.666731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.668451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.719717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.776953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 319/634] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:50.782245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.794617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee3777-d738-4f9d-8c7f-8eb01f9dacf5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.838453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.839061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.839373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.839707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.840372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:50.840726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.842130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.852306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 316/635] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:50.858592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.897752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5d4a9-5a5f-4c99-8510-3afbc588e123 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:50.938148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:50.938897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:50.939266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:50.939726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:50.951413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:50.953081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:50.969887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:50.970879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:50.972346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:50.972346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:50.972346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:50.972346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 320/636] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:50.973311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:50.979649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:50.983825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.094085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a313e5-baa3-4c06-af61-671766a083e1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.134274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.135555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.136065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.136545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.136835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b270a8a-2366-4d95-8b65-d8453082481c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.137943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:51.138601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.141065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.156434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 321/637] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:51.157602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.173809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.174964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.175265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.175541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.188644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.190127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-96d39f9f-b83c-4e60-b064-17235d8d5c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:51.206383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:51.207696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:51.208583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:51.208583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:51.208583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:51.208583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 317/638] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:51.209275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c79c6-9680-4f02-8708-0009789fa6ba None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.209560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c79c6-9680-4f02-8708-0009789fa6ba None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.209836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c79c6-9680-4f02-8708-0009789fa6ba None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.215120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 318/639] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:51.215120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c79c6-9680-4f02-8708-0009789fa6ba None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.223846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.274710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b93f-ef14-486e-a6c0-385dba2f464e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.310459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.311091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.311402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.311679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.322206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:51.322879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.327303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c79c6-9680-4f02-8708-0009789fa6ba None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:51.328745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:51.329715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:51.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 322/640] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:51.331875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.373085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.373626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.373908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.374182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.381244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.382685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b1c7be7a-5b13-42e5-bc7a-4a64b9396c0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:51.395526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:51.396695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:51.397774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:51.401296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 319/641] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:51.402402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.402694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.402976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.421790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:51.422972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:51.447106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979251ec-657a-4170-9261-abf79e3fc508 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.471297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.503456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 320/642] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:51.503456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.519255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.519255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.519255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.519255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.519803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:51.520115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.521637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.540016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 323/643] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:51.541146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.626170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.644578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f667416-02a1-4368-97a9-265c00449e9f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.666296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b9dc4-e392-4571-bd7e-e725f38f73d6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.686634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.687511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.688185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.688469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.689135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:51.689442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.691520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.708143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 321/644] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:51.709212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.719884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.720495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.720778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.721080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.722323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:51.722637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.725045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.742795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 324/645] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:51.743755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.779723 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.818587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f8812-95f3-4f90-9181-5c281997612f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.859422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50334b4-d8fd-4400-9315-34b23fb44e19 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:51.864219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.864861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.865140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.865416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.873711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.875177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:51.887878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:51.889770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:04:51.891369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:51.894364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 322/646] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:51.895101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:51.902456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:51.903092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:51.903404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:51.903731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:51.911943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:51.917076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:51.918542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:51.934302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:51.935278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:51.936176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:51.936176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:51.936176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:51.936176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 325/647] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:51.936398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.010225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6acb62-606b-44e5-846f-80eabaffbad6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.030393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.046581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d591be-2b25-412d-a242-b60e8adca0d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.050901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.051631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.051983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.052324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.053114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:52.053497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.055000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.067327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 323/648] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:52.067327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.102268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.103035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.103677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.104031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.121154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:52.129996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:52.131172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:52.132195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:52.132956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 326/649] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:52.137598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.170485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.201376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f3ffc-cc61-4d1c-966c-83712cd1ab33 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.240900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.241636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.242018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.242298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.243059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:52.243397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.244979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.264226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 324/650] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:52.265143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.266147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd80fc2-ff22-4ffc-b7a9-2ffa4d250581 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.312695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.313293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.313575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.313846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.314490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:52.314789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.317537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.330630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 327/651] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:52.332279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.343860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.392493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c6dc-f189-463b-9847-24d891e6c0f2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.436587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.437705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.438117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.438503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.439681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:52.440749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.442885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.444716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.454802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 325/652] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:52.456293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.467731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e846-a320-482d-89d3-574987ccab13 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.511175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.511813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.512099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.512377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.573969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c800f2f-61c3-41f2-a7b0-6d07880b1a6b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.585815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.587323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1185c4-42ac-4acd-b370-81d9321076ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.595549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.613818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.614789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.615255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.615822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.624236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.625912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:52.639484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:52.640697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 16:04:52.641985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:52.644792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 326/653] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:52.645734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.705126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 328/654] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:52.706526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022f19-0584-427c-874a-7a707eaf22eb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.706914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022f19-0584-427c-874a-7a707eaf22eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.707338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022f19-0584-427c-874a-7a707eaf22eb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.708848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 329/655] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:52.713360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022f19-0584-427c-874a-7a707eaf22eb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.714493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.763555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d0f8-96af-425a-97b7-b13740982ff9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.802012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.802630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.802918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.803204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.814626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.816164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.825795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022f19-0584-427c-874a-7a707eaf22eb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:52.833316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:52.834293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:52.835213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:52.835213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:52.835213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:52.836168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.839758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 327/656] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:52.840538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:52.875331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.876216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.876506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.876776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.878328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:52.878954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:52.880879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca084f-a645-4ba6-aa6b-c1f107f4fbde None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:52.904513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 330/657] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:52.910232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.910522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.910801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:52.929200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:52.930414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:52.956621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecb865-6d4c-41fb-b284-fac31634075b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:52.966101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:52.993556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:52.994185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:52.994471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:52.994936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.002209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 331/658] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:53.003615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.017418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.018941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:53.034436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:53.035510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:53.036425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:53.036425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:53.036425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:53.039587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 328/659] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:53.040403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.137931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.149367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98833-1a8d-41e6-9f35-a7c8f03a9d6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.150590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f437658-9266-4da8-ac2c-dac5973cc841 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.187213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.187870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.187977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.188285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.198228 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.199894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.200177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.200448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.201496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:53.202017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.204939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.223061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 332/660] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:53.223887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.229998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.283803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.285267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.343157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3719e-6dd2-4004-93e0-43f130e0af1e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.379791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.380388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.380668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.380941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.381574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:53.381876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.383264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.401950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 333/661] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:53.403260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.403387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 329/662] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:53.404177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.420901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.474270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.517134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ff221-51fb-4b83-9a31-f0fd09279b8e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.520544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02ba8-6913-45ed-a6b9-174c2c1546e8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.556002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72167-e017-498d-bb26-16272f928080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.556202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.556633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72167-e017-498d-bb26-16272f928080 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.556738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.556916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72167-e017-498d-bb26-16272f928080 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.557015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.557189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72167-e017-498d-bb26-16272f928080 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.557297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.566668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f72167-e017-498d-bb26-16272f928080 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.568163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:53.568457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f72167-e017-498d-bb26-16272f928080 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:53.572982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:53.573932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:53.574790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 330/663] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:53.575556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-01f72167-e017-498d-bb26-16272f928080 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:53.583049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:53.584315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 16:04:53.585248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:53.587363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 334/664] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:53.589056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72167-e017-498d-bb26-16272f928080 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.680834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.692180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7a84-ff04-4cef-b86f-97d49e0eb663 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.710337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72167-e017-498d-bb26-16272f928080 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.718886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.735383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958857ef-d531-4542-9070-76c168ce16df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.735939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958857ef-d531-4542-9070-76c168ce16df None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.736488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958857ef-d531-4542-9070-76c168ce16df None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.736764 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958857ef-d531-4542-9070-76c168ce16df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.744440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958857ef-d531-4542-9070-76c168ce16df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.745886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958857ef-d531-4542-9070-76c168ce16df None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.750938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.751599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.751888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.752168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-958857ef-d531-4542-9070-76c168ce16df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:53.757541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:53.758420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:53.759377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 331/665] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:53.759887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958857ef-d531-4542-9070-76c168ce16df None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.770209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.772118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:53.788776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:53.789773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:53.790656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:53.790656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:53.790656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:53.790656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 335/666] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:53.795809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.875475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958857ef-d531-4542-9070-76c168ce16df None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.911709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc076-c3b6-4fea-976e-805b4f344d10 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:53.913548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.914735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.914735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.914735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.915393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:53.915701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.917066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.925661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 332/667] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:53.926485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:53.947001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.956933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:53.957781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:53.958265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:53.958539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:53.959310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:53.959622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:53.961063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:53.977164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:53.978496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 336/668] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:53.979612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.045777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08297c-d00b-4876-903b-54e46f1b041a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.085060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.085910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.086378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.086805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.087707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:54.088164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.088453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3abbf-5c02-4fb6-a608-b3361d40f245 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.089920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.103497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 333/669] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:54.104571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.129573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.130182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.130464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.130724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.131408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:54.131717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.133209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.145286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 337/670] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:54.146132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.197954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.214241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fe07a-9c4e-41af-a6e4-460115c6b163 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.215807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.256279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.256709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae4ed6-b6b4-4d71-99a0-057843b0c6df None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.256912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.257199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.257478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.267798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.269837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:54.285158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:54.286192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:54.287105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:54.287105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:54.287105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:54.287105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 334/671] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:54.287105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.296142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.296683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.296962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.297231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.365502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.366976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccbca6-d4fa-40a3-aa3f-63e6ffb33e9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.386868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534688-7d42-4452-8424-0b922130e60b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.411972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.420600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.421201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.421474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.421742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.422450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=de5581201518429191f74cf8af19fb58) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:54.422753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.424191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.429914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 335/672] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/role_assignments?user.id=de5581201518429191f74cf8af19fb58&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:54.430869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.456983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.478687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 338/673] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:54.478687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98223dd-8553-4d60-9e25-b7c27849b980 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.478687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98223dd-8553-4d60-9e25-b7c27849b980 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.478687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98223dd-8553-4d60-9e25-b7c27849b980 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.482063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 339/674] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:54.483037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98223dd-8553-4d60-9e25-b7c27849b980 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.564131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d3edf-741e-4d07-90a1-a3d29ff74162 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.606165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.606805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.607094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.607400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.613059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98223dd-8553-4d60-9e25-b7c27849b980 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.614666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.616180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:54.629693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:54.630627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 336/675] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:54.632256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.668301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.669415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.670744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.671188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.672019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:54.672474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.673994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fce06-4c0e-4ffb-8b67-2113a8d96545 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.693638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 340/676] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:54.693638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.693638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.693638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.699687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.712692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:54.714005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:54.723531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.748334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ad6b6-79f1-4841-bcd3-bad73fbb046f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.786182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.787047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.787530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.787789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 341/677] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:54.788120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.788680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.807078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.808626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:54.824361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:54.825532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:54.826635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:54.826635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:54.826635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:54.829946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 337/678] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:54.831388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:54.919498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3180e1f-247a-4e02-908f-5c9cb38380c8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.957468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:54.958080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:54.958417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:54.958635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:54.959381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:54.959919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:54.961131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:54.962946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:54.973719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408e4f-7ed5-42a2-9d2e-6846ef9d6cd6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:54.980446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 342/679] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:54.981401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.013627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.022663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.023540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.024007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.024433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.035766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.037807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:55.056092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:55.057281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:04:55.058440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:55.065880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 338/680] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:55.065880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.109420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01a9bc-a138-4b1c-82b6-f528f0e9f71f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.146912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.147591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.148082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.148082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.148683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:55.148984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.150358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.166958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 343/681] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:55.166958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.178143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ab464-f33c-4db2-9c87-8f9c3b915c4f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.216949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.217623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.217848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.218089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.218819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=c34f7dd7c12a4262a6ace2f07b6475c1) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:55.219128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.220622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.226878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.227013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 339/682] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/role_assignments?user.id=c34f7dd7c12a4262a6ace2f07b6475c1&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:55.227863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.296373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b93b-5fd4-4ec3-b169-85e62585dc22 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.311957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.345379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.346830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.347613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.348084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.349350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:55.350006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.352635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.371798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54b58-5b27-4c60-87f3-d05b2a9059b0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.381001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 344/683] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:55.382137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.421278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.422003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.422512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.423052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.442272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:04:55.450289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:55.451277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:55.452155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 340/684] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:55.452837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.514523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832b7c-033c-416c-85c1-2830714af852 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.534775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.558483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.559518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.560021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.560490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.569166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.571294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:55.585883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:55.586847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:55.587740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:55.590281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 345/685] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:55.591298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.591630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777ddc-7fa5-4ff6-a1b7-7ad9d5903d3c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.610291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.639032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.639603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.639890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.640170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.716673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c1def-bccb-44c3-adcb-391ba2b16d46 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.740260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.741737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.758002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.758937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.759469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.760062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.761022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:55.761523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.763172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.773560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 346/686] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:55.774660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.816746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.881263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 341/687] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:55.882525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:55.896309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7ead7-b832-456a-b79e-ec48ab679fe6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:55.901630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:55.938860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:55.939962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:55.940447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:55.940911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:55.941915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:55.942521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:55.944180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:55.960403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 347/688] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:55.961421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.006888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e34b62-1176-4352-acfe-68930bbf7227 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.056414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.057414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.057894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.058816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.073398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.075758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.078617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.087414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92a874-5729-4a9f-a2a4-9ae506f8fc2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:56.102687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:56.103713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:56.104575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:56.104575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:56.104575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:56.107047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 342/689] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:56.108852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.134038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.134825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.135341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.135726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.146636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.148211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:56.164596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:56.165596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:56.166474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:56.166474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:56.166474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:56.166474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 348/690] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:56.166912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.195341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.236648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc227a35-36bf-48ea-a732-8af24477c457 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.284593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.285451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.286129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.286550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.287880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=393f32c7fcdf49219d155bb22fd3d048) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:56.288445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.290033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.299166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 343/691] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/role_assignments?user.id=393f32c7fcdf49219d155bb22fd3d048&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:56.300190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.314348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b200c-7af5-4b6a-ad11-970d24b47914 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.417834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.418709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.419140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.419579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.420382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:56.420857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.422412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e00c36-2ae7-4fbd-9776-9b1aa7de4bdd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.427340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f89e4-525c-467b-9e70-f20d8805651a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.448592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 349/692] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:56.450090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f3c27-217c-4d70-8534-6b3af14db46d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.450379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f3c27-217c-4d70-8534-6b3af14db46d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.450652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f3c27-217c-4d70-8534-6b3af14db46d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.457099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 350/693] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:56.458010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f3c27-217c-4d70-8534-6b3af14db46d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.464776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.467300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.472747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.473608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.474030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.475082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.475933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:56.476753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.478105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9109454-592a-47d9-b10c-14b6f83dd0ac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.492505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 344/694] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:56.493767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.495522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.495820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.523969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:56.525513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:56.581404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f3c27-217c-4d70-8534-6b3af14db46d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.618878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 345/695] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:56.620196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.636530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.637540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.638023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.638524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.653799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.655283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:04:56.679706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:56.680665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:04:56.681601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 351/696] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:56.688408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.779472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.818751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aeeda-5499-428b-b8a5-312a7948b6e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.855897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.856781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.857231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.857655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.858538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:56.858989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.859912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76adf3-f38a-4194-9e67-9fd05adda00e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:56.860859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.881876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 352/697] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:56.888377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:56.896028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:56.931270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:56.932034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:56.932320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:56.932600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:56.942426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:56.944200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:56.960407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:56.961507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:56.962513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 346/698] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:56.962996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.011770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562991d7-71ff-4d5f-945c-e2aedf12f065 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.038535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.051690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.052652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.053108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.053577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.092901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ea168-6e09-4025-9a4d-30081c82f47a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.129539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.130485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.130978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.131441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.132176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.133673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.144428 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:57.145487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:57.146725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:04:57.147701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:57.148435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.150303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 347/699] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:57.151370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.201930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.260287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4549db-4ab0-439c-956b-f54c756600b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.280326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 353/700] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:04:57.281210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.301993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.302935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.303398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.303814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.304650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:57.305099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.307070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.313366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.321957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 348/701] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:57.322982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.410472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4f64-6db6-4e7c-b0a0-9c3d6219fee4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.447569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168294e9-0922-4008-8fbf-2230424ed430 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.448539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.449345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.449815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.450087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.450726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:57.450835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.451034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.452507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.459643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 354/702] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:57.459643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.488776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.489412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.489693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.489964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.500533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.501956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8507c228-79dd-4d9a-b835-68ac453ab644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:57.518211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:57.519317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:57.520267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:57.520267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:57.520267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:57.520267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 349/703] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:57.520497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbec50-a904-43cd-a02f-98f0ceda1c23 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.521321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbec50-a904-43cd-a02f-98f0ceda1c23 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.521670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbec50-a904-43cd-a02f-98f0ceda1c23 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.523432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 350/704] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:57.528046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbec50-a904-43cd-a02f-98f0ceda1c23 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.566517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.582131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdb211-2047-4d7f-8edc-efaf627ac00f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.635534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.636653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.637078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.637486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.638561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=058f62fc005a41b08ee30fb42192e31a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:57.639364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.641494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.652777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 355/705] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/role_assignments?user.id=058f62fc005a41b08ee30fb42192e31a&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:57.653863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:57.670779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbec50-a904-43cd-a02f-98f0ceda1c23 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.713150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.713735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.714021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.714301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.714962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:57.715289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.716726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ddc07-1707-4315-bf17-83da3344a5fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.732103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 351/706] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:57.736830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.738107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.738107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.804079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:57.805524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:57.858253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:57.859632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373a8e-8525-4a83-94c5-9d9cf0874564 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:57.921950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:57.922936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:57.923522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:57.924031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:57.925764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:57.926484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:57.928709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:57.946493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 352/707] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:57.950230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 356/708] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:57.960382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.113555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9a5f4-75cf-4af9-98bc-0f4f1b872e80 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.169184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.170198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.170805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.171313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.179347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.181402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:04:58.194908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:58.196087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:04:58.196981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:58.199379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 353/709] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:58.212409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.229753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:58.280324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.303917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:58.333211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38237-d118-46d9-a1b0-fe3305823f24 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.376415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.377783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.378220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.378634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.379380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:58.379693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.381327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.397579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 357/710] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:58.403265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.421812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8a17-e83d-45e2-9f18-8951d2c6f27e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.458330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.459082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.459463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.459817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.472163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.473939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:04:58.491480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:58.492647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:04:58.493711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:04:58.493711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:04:58.493711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:58.498245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 354/711] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:04:58.499494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.569163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:58.599617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:58.618105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a4df-38eb-4a6a-a304-6285d77abf15 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.637569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15bcca-a2d3-4e63-b4d7-f22bb3a8f2bb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.667921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.669095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.669615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.670084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.671007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:58.671536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.671644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.672206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.672491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.672819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.673473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:58.673786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.674727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4536b1-76df-46cd-98cd-53886a34d3f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.675696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39d8a-3dac-4b73-b200-cade90bc87e0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.681893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 358/712] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:04:58 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:58.683896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530050e-5753-429e-99e6-9d44395aa031 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.684389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530050e-5753-429e-99e6-9d44395aa031 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530050e-5753-429e-99e6-9d44395aa031 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.687543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 359/713] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:04:58.691900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 355/714] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:04:58.693425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.693713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.693991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.696242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530050e-5753-429e-99e6-9d44395aa031 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:58.715453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:04:58.716886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:04:58.848784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 356/715] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:04:58.888858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530050e-5753-429e-99e6-9d44395aa031 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:58.942580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:58.943185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:58.943544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:58.943825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:58.944470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:58.945941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:58.961781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:58.982502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 360/716] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:04:58 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:04:59.286122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:59.344374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:59.400374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bb7fc-d0e6-4a29-a768-e504b12721df None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:59.464921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:59.465629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:59.465909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:59.466184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:59.466775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:59.468251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:04:59.484884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:04:59.485976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:04:59.488730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 357/717] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:04:59 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:04:59.493132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:59.653867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350252-778e-42ea-acaa-a4caace4e3ce None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:59.713054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:59.713666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:59.713947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:59.714223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:59.714860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:59.715160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:59.716555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274c9a4-3242-4cd4-b0c7-0bc7250ec909 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:59.727489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 361/718] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:04:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:04:59.732938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:04:59.878119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8034c-797b-4f67-8b57-d0abf2acc75d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:04:59.899313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:04:59.930805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:04:59.931496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:04:59.931783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:04:59.932058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:04:59.933515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:04:59.933824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:04:59.935255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af503e4-88db-4d0d-8e2c-490d3f9f25f0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:04:59.944864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 358/719] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:04:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:00.195063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17735c-8ddd-47c3-afd9-f6f4a0de310d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:00.195472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17735c-8ddd-47c3-afd9-f6f4a0de310d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:00.195654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17735c-8ddd-47c3-afd9-f6f4a0de310d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:00.197225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 362/720] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:00.202371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1511b0-75a0-4489-8201-67302ee4ea76 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:00.202662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1511b0-75a0-4489-8201-67302ee4ea76 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:00.202940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1511b0-75a0-4489-8201-67302ee4ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:00.223891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0e1511b0-75a0-4489-8201-67302ee4ea76 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:00.225133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e1511b0-75a0-4489-8201-67302ee4ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:00.321319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 359/721] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:05:00.415430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b651-6a42-451d-8def-588ecd6aec55 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:00.415751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b651-6a42-451d-8def-588ecd6aec55 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:00.415975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b651-6a42-451d-8def-588ecd6aec55 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:00.417566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 363/722] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:00.422472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:00.422760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:00.423038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:00.447611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:00.448782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:00.547972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 360/723] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:05:00.627505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b651-6a42-451d-8def-588ecd6aec55 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:00.707490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:00.769685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b651-6a42-451d-8def-588ecd6aec55 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:00.816614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:00.817631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:00.823916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:00.824439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:00.825329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:00.826932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d3c04-87b0-4181-92d2-a4667a16bae2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:00.848905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 364/724] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:00 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:00.992292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.046417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:01.121753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9a57f-f51c-4ef6-ad14-685a0fdb3fd4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:01.154829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c30e5-9619-4bbe-b9e7-d3b3feb1e583 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.155503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c30e5-9619-4bbe-b9e7-d3b3feb1e583 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.156017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c30e5-9619-4bbe-b9e7-d3b3feb1e583 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.157852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 365/725] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:01.163493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.164150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.164633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.185032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:01.186583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:01.188518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:01.189172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.189456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.189799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.202705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:01.208985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:01.210104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:01.211114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:01.213843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 361/726] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:01.221743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.280854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 366/727] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 16:05:01.382645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:01.388723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836c67-5f09-4c2f-b36f-857ad3422c07 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:01.441084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:01.441698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.441989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.442265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.442938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:01.443304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:01.444640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07d0e-a9b0-4c00-81ee-10f0a4821bfd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:01.459274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 362/728] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:01.464465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.543137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:01.578890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2481-04fa-4aa7-a07c-76ff98fa9451 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.579201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2481-04fa-4aa7-a07c-76ff98fa9451 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.579476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2481-04fa-4aa7-a07c-76ff98fa9451 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.581549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 363/729] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:01.586673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.586957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.587258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.609855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb57ac-cd3d-4ff8-9554-96da10200342 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:01.639407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:01.640628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:01.651823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:01.653082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.653572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.654050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.666959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:01.668765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:01.684795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:01.685754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:01.686708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:01.686708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:01.686708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:01.688698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 367/730] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:01.694306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.731249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 364/731] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:05:01.731249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.857795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:01.865125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8caa8d-4d16-41de-ab73-ebcf90d8921d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:01.891997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:01.914791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940acaf5-a0b1-4f86-9f24-79fbf9d6c958 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:01.930796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:01.931727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.933642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.934125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.951854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:01.958225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:01.959201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:01.960311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 368/732] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:01.965597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:01.982144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:01.983139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:01.983682 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:01.984185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:01.985035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:01.985777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:01.990276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.008205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 365/733] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:02.014442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.134250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac0f5-c545-4b08-83c0-7893366d736d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.153992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7f5f-837d-4c85-97a0-624d573a3afc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.171726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.172334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.172614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.172886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.173557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:02.173859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.175402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.184546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 369/734] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:02.185347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.194300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.195295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.195798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.196312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.204643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.206589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.209054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:02.220324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:02.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:02.222821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:02.225388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 366/735] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:02.226405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.255645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.311262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db97bc7-be25-409a-832c-994f325ff713 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.339160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363c0b-4d21-4753-9ed0-a65173328cf2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.349523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.350150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.350437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.350707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.361382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:02.367169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:02.368224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:02.369042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:02.373106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 370/736] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:02.373769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.378837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.379508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.379796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.380073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.391521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.393433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ede9e53d-3019-4b3d-938d-308f95655488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:02.408077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:02.409718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:02.410590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:02.410590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:02.410590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:02.410590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 367/737] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:02.411454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ef84-05f1-4d6d-b383-9efdfc1ef9d0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.411454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ef84-05f1-4d6d-b383-9efdfc1ef9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.411578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ef84-05f1-4d6d-b383-9efdfc1ef9d0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.415263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 368/738] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:02.415263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ef84-05f1-4d6d-b383-9efdfc1ef9d0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.468096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.479015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.490979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e009c0-bee6-466c-a25e-ccf22da17a36 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.526027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.526520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ef84-05f1-4d6d-b383-9efdfc1ef9d0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.526813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.527159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.527467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.528226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:02.528537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.529977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.553169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 371/739] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:02.553169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.567320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.567931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.568209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.568481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.569141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:02.569446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.570801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4daa-a536-4a45-a8e3-cb8147b96ea7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.584920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 369/740] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:02.584920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.584920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.584920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.609958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:02.611989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:02.691123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 370/741] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 16:05:02.702851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbcdfe-1590-4796-8837-d92389aaccb0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.704173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.748980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.750639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.751072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.751517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.752977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:02.753460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.754992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.764569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.771736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 372/742] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:02.772837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:02.777908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:02.833411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55a91-f5eb-4d77-a0a6-7f36ceb018a9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.869722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.870571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.870995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.871437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.894505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058bef2-28b0-4238-9a55-2db63653975c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:02.937235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:02.938123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:02.938510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:02.939522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:02.951340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.953101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.956329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:02.957999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:02.973605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:02.974776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:02.975866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:02.975866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:02.975866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:02.979588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 373/743] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:02.980441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.011576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.039198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.062038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 371/744] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 16:05:03.064154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.094981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b7f58-2ecf-4d14-9296-e436084e5b42 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.130852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.131504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.131786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.132082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.139833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.141284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:03.152715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:03.153696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 374/745] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:05:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:03.154640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.181369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7207f3f-7908-4bee-b25a-6bf4def04888 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.215759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416678-502b-4919-9181-c77c905882fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.216275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416678-502b-4919-9181-c77c905882fd None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.216562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416678-502b-4919-9181-c77c905882fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.216832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416678-502b-4919-9181-c77c905882fd None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.217470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416678-502b-4919-9181-c77c905882fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.218829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416678-502b-4919-9181-c77c905882fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.235121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 372/746] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:03 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:03.237235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416678-502b-4919-9181-c77c905882fd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.255555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.266638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561cae-f1a5-47ea-9ceb-0adba4e68664 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.279553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.316660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.317436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.317721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.317996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.318650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:03.318957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.320654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.339857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 375/747] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:03.339857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.365415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416678-502b-4919-9181-c77c905882fd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.405549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.406241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.406484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.406767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.419999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.421769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5d0bc9cb-453f-495b-8289-079dad44e3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:03.441346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:03.442325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:03.443212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:03.443212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:03.443212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:03.443889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 373/748] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:03.445402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfa38a-d18c-4e55-ac2a-e582cf3a166a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.445753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfa38a-d18c-4e55-ac2a-e582cf3a166a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.446146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfa38a-d18c-4e55-ac2a-e582cf3a166a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.447775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 374/749] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:03.452359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfa38a-d18c-4e55-ac2a-e582cf3a166a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.458390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101a59c-d729-4bcf-9411-eed5dfcc9c9d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.497689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.498316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.498559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.498835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.499659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:03.499971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.501881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.521500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 376/750] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:03.522339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.530698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.552510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.576093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfa38a-d18c-4e55-ac2a-e582cf3a166a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.615165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.615877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.616155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.616433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.624730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.626281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.636694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edbe6-f18d-4203-8b39-116543ed9270 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-895e6afc-c9e3-4bfa-aaf2-84e5620967a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:03.638761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:03.639757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:03.640722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:03.645741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 375/751] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:03.647080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.647397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.647674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.666303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:03.667372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:03.677340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.677944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.678221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.678495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.679141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:03.679476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.680844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.693773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 377/752] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:03.694591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.745259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 376/753] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:03.746431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.804481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.827618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4957697-5f94-4d27-bd87-ff315a7d671d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.831426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:03.870389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.871909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.872407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.872867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.887968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a7e8c-8600-4593-a9d2-20f5ff474820 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:03.888720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.891751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:03.912031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:03.913065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:03.913980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:03.913980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:03.913980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:03.916626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 378/754] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:03.917634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:03.941345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:03.942585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:03.943023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:03.943507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:03.944390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:03.944871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:03.946974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:03.971105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 377/755] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:03.972318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.048287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5c6c-a229-4ed8-9c82-9503531a0ea5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.088165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.088713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.088993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.089271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.092136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65a3ba-f6aa-4954-8300-3160eccd8f4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.092282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.097597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.099588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.106847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:04.113711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:04.114609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:04.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:04.119613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 379/756] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:04.120558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.132428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.133040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.133323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.133600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.134258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:04.134567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.135961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.152295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 378/757] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:04.152295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.232599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696fb4a-6eea-4597-83a0-21fe33f765d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.265322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbdb30-278c-4be4-8c79-57ab6ba34969 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.275715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb19-5407-43db-bc82-58441967520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.276331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb19-5407-43db-bc82-58441967520e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.276614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb19-5407-43db-bc82-58441967520e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.276888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb19-5407-43db-bc82-58441967520e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.311000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.311658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.311938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.312264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.312785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.314144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:04.323874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:04.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:04.325739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 379/758] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:04.326526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.352552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.363026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875abb19-5407-43db-bc82-58441967520e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.364718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875abb19-5407-43db-bc82-58441967520e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.375313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.446077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d9884-bf21-4bc5-a4c4-89fb3e837ebf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.475859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 380/759] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:04.475859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb19-5407-43db-bc82-58441967520e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.486657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.487292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.487577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.487851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.488541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:04.488843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.490261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.501802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 380/760] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:04.503089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.584627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.592027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb19-5407-43db-bc82-58441967520e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.612163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.618600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f66a1-9767-4317-8389-dae741d0a593 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.636473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.637206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.637395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.637673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.645739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.647384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:04.659750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:04.660820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:04.661898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:04.664915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.665528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.665636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 381/761] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:04.665788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.666058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.666532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.666700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:04.666979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.668366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.673204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 381/762] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:04.673838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.778679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a515144-06fc-45bb-b9a1-b4041869c096 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.781335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441fb98-4077-4dc5-ae0c-0da8beac7aee None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:04.813657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.814262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.814616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.814891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.815662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=058f62fc005a41b08ee30fb42192e31a) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:04.815938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.817348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.819215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:04.820552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:04.820985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:04.821410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:04.822795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 382/763] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/role_assignments?user.id=058f62fc005a41b08ee30fb42192e31a&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:04.823621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.855247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:04.856786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:04.871125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:04.876501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:04.878287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:04.879391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:04.879391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:04.879391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:04.886526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 382/764] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:04.888153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:04.914559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:04.964623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aec9b9-b467-40ea-b307-489520de5062 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.003306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.004835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.004835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.004835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.005429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:05.005745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.007175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.025025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 383/765] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.025943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.026157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2012f-e115-41f3-aa51-eb9558586a5e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.065705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.066316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.066601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.066877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.067655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:05.068033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.069478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.081474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 383/766] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:05:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.082373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.115683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.136330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f1ac-6e65-4004-a5ca-819e6bae1fca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.165528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.173726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.174953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.175259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.175539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.176245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:05.176551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.177978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.194805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 384/767] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.194805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.208341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829444e6-f98d-4e7b-910a-175b935c4978 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.245799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.246463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.246748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.247024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.248967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=5ce87b2207504e839bcd544faa3aa7ad, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.250411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.271269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 384/768] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:05 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:05.271269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.321182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1979-e755-4afa-a608-5424b5f08310 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.374547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.375340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.375699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.376047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.376928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] RBAC: Adding query filter params (interface=public, service_id=f2b730391b5e4ffb99ccd63cbaf86572, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:05.377837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.379337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50c040-fae9-411d-8a43-1b855de11f6f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.394315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 385/769] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/endpoints?service_id=f2b730391b5e4ffb99ccd63cbaf86572&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.394315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18eb32-f5bd-4c4f-9111-2897f141954e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.394315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18eb32-f5bd-4c4f-9111-2897f141954e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.394315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18eb32-f5bd-4c4f-9111-2897f141954e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.398716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 386/770] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:05.400800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18eb32-f5bd-4c4f-9111-2897f141954e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.416350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.428997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377be9e5-b6a3-4b26-91cb-088011838cb5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.463793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.476154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.477126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.477416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.477700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.488953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.491356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-12a45261-cc18-433f-a82f-7d08163f2a53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:05.507058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:05.508046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:05.508968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:05.509400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 385/771] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:05.513371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.513371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.513371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.536844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18eb32-f5bd-4c4f-9111-2897f141954e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.537374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:05.538072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:05.573733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.574333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.574621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.574894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.575532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.576951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.591249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 387/772] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.591249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.623940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 386/773] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:05.741873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:05.780780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c20d2-8d62-4118-a5a8-3164ea44d659 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:05.832039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:05.832851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:05.833087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:05.833298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:05.834073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:05.834387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:05.836303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:05.850000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 388/774] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:05.856136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.948997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:05.983045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6ac42-89a9-47cf-8beb-4ab5c2d0e334 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.014023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:06.040285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.043771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.044259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.044729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.046655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=53087055f7e0417b99552e16781f4542) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:06.047588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.050022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.063339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 387/775] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/role_assignments?user.id=53087055f7e0417b99552e16781f4542&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:06.103679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae46683-138d-409b-809b-2c9643e1b9ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.151488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:06.159673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.160314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.160591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.160862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.161451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.162805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:06.175381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:06.176416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:06.177637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 389/776] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:05 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:06.184804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:06.309310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344556a0-65a1-4d46-959a-f64604da1dd8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.344610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.345213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.345491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.345912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.347033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:06.347413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.348852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.354058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 388/777] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:06.360395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:06.494069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11279c7-8d07-4957-b69b-5dc395b5e74a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.540110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.541561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.542041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.542508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.543361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:06.543859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.545424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.555389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 390/778] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:06.561345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:06.638970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:06.716888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858c0a-d777-4ee7-bc0d-f8e9211a93a6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.757652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.758268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.758550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.758828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.759565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] RBAC: Adding query filter params (interface=public, service_id=5dc573508eaf4dfba95498dcd7e92847, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:06.759878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.761256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6810d65-1ff8-46e4-9d9b-02e947c61e5a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.770061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 389/779] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:06 2022] GET /identity/v3/endpoints?service_id=5dc573508eaf4dfba95498dcd7e92847&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:06.775787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:06.869674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:06.917478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d491e-6a1a-4282-b86d-c4a10f5f718e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:06.969467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:06.972295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:06.972585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:06.972862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:06.973540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:06.974907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec8e3-7c18-4f64-8077-a109863cba4c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:06.984217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 391/780] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:06 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:07.138565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d815d-81ee-4acc-98e5-e91184d976ba None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.139065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d815d-81ee-4acc-98e5-e91184d976ba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.139143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d815d-81ee-4acc-98e5-e91184d976ba None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.140783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 390/781] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:07.146548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f5e9-747b-4d18-b4b4-79624d2ab1d8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.146838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f5e9-747b-4d18-b4b4-79624d2ab1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.147102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f5e9-747b-4d18-b4b4-79624d2ab1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.177857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6386f5e9-747b-4d18-b4b4-79624d2ab1d8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:07.179083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6386f5e9-747b-4d18-b4b4-79624d2ab1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:07.259174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 392/782] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:07.489227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83577-b510-48bb-a813-8799a0d0c975 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.489605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83577-b510-48bb-a813-8799a0d0c975 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.489789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e83577-b510-48bb-a813-8799a0d0c975 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.491360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 391/783] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:07.496787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bab42-9846-430d-9998-c111339f876f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.497088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bab42-9846-430d-9998-c111339f876f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.497369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bab42-9846-430d-9998-c111339f876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.517103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-190bab42-9846-430d-9998-c111339f876f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:07.518356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-190bab42-9846-430d-9998-c111339f876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:07.585704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:07.592019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e83577-b510-48bb-a813-8799a0d0c975 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:07.614398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 393/784] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:07.772552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e83577-b510-48bb-a813-8799a0d0c975 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:07.794447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:07.821270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:07.821918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:07.822204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:07.822480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:07.843141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:07.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:07.851088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:07.852152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 392/785] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:07.857526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bab42-9846-430d-9998-c111339f876f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:07.963677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.002791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bab42-9846-430d-9998-c111339f876f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.050117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.050651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.050929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.051196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.051890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:08.052193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.060882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.077625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 394/786] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:08.081892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.166820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.197822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63422c6-47eb-4e37-82bc-1aedefc755f8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.264242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.264980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.265270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.265646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.268876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de99541-12c8-4978-a627-791dd2321246 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.276817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5b442621-4c36-44b8-8eac-54984d8eaf1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:08.282462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:08.283943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:08.285172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 393/787] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:08.285854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709e988-247a-457d-875e-907215826929 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.286144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709e988-247a-457d-875e-907215826929 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.286290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709e988-247a-457d-875e-907215826929 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.287978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 394/788] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:08.292168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709e988-247a-457d-875e-907215826929 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.315068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.315712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.315992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.316808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.331151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.332215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.335322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fad2045c-ea20-49dd-bee2-b593b4f953ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:08.355540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:08.357916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 395/789] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.363046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.379442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:08.380627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:08.408281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709e988-247a-457d-875e-907215826929 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.444897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.445505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.445791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.446068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.446737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:08.447049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.448503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adab-9bf1-4093-90c2-a00978f9821f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.457653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 395/790] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:08.458891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977be9-d472-4a2f-ad20-21cb824fcdcb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.459181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977be9-d472-4a2f-ad20-21cb824fcdcb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.459487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977be9-d472-4a2f-ad20-21cb824fcdcb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.463273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 396/791] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:08.463273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9977be9-d472-4a2f-ad20-21cb824fcdcb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.474785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 396/792] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:08.475949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.515493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.583891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.594982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9977be9-d472-4a2f-ad20-21cb824fcdcb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.618183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba14bc7-ccd7-4bfd-82fa-dff9cb314649 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.646863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.647814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.648320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.648795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.649732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:08.650304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.652011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8c7f3-2465-4863-bad9-d417c9c290cd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.661856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.662528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.663367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.663657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.666918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 397/793] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:08.668531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.669055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.669629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.675037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.676524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.698293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:08.700100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:08.784371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 398/794] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:08.788258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:08.822219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:08.823251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:08.824131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:08.824131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:08.824131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:08.828168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 397/795] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:08.829032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.885244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.905204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177456cd-34e7-4b3b-a21e-aa7d4196bf68 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.917962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:08.938637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df051-534b-4047-8c25-6cb3a642fc73 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:08.944610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.945548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.946027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.946495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.954278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.956207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:08.968587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:08.969548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:08.970462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:08.974003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 399/796] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:08.975375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:08.975553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:08.976185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:08.976472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:08.976750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:08.977334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:08.978706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:08.988488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:08.989514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:08.990331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 398/797] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:08.991115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.088642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09c84-5456-4e2f-a705-169695a65ba1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.104150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf50cd-f90a-4981-93c5-69f3b22e8ff8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.126805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.128384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.128881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.129605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.130718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.131406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.133603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.139668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.140208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.140486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.140756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.141483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.141782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.143904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.155440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 400/798] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.156844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.162104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 399/799] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:05:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.162928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.167993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.168976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.267537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a59e-a8be-4e29-ae45-159b2cb4c845 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.280960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173919e-340a-4578-ab25-7d3fcc43df65 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.304057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.304845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.305133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.305407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.306162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.306584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.308011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.317728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 401/800] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.317728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.321075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.321709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.321991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.322266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.333444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:09.338238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:09.339169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 400/801] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:09.345071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.399296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.407178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.436853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd22a-f73f-4a73-a558-375a9e0a7971 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.450302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3920383-92ca-4bfc-b452-8b347ab8f3ef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.472099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.472728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.473012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.473293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.480445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.481899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.486915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.487572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.487854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.488126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:09.494186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:09.495359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 402/802] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:09.497068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.503034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.504560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:09.520226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:09.521374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:09.522458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:09.522458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:09.522458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:09.522458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 401/803] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:09.522726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.602210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf985f1e-1a2c-4867-9897-5d464cc4715e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.625092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1768-d127-4047-a54e-4a28f726c8b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.632270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.639022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.639665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.639947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.640228 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.640454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.641337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.641647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.643719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b405435-fd0e-4e89-9659-6592d3f2f0b2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.649832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 403/804] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.651009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaabc79-3ada-45bc-8d31-6e343ee456c5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.651327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaabc79-3ada-45bc-8d31-6e343ee456c5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.651604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaabc79-3ada-45bc-8d31-6e343ee456c5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.652838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 404/805] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:09.653529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaabc79-3ada-45bc-8d31-6e343ee456c5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.669516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.670201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.670484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.670756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.671448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.671751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.673174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.681386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 402/806] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.682198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.764669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaabc79-3ada-45bc-8d31-6e343ee456c5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.790703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7cf0f-2adf-4f51-a7da-402181866096 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.799841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.800450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.800738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.801030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.801753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.802060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.803520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.820837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 405/807] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.821692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.824122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.824689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.824972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.825246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.826149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:09.826462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.827929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d743d8f-669f-4d45-bf60-ae3021f5b45d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:09.839899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 403/808] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:09.841105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.841384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.841660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.859631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:09.860822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:09.879804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.892112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:09.932114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444864c-39b9-4e87-b446-deb60a5bd164 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:09.933826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 404/809] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:09.934670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:09.988304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:09.988304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:09.988304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:09.988304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:09.989707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:09.991266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.019311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 406/810] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:10.020280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.058771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22b33c-48d0-40a7-8853-1601e89e9830 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.109592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.110328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.110607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.110880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.124070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.125624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:10.143474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:10.144499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:10.145389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:10.145389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:10.145389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.149609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 405/811] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:10.150950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.169192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323d4-cdee-463d-a9bb-615540213148 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.171809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.180697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.213249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.214118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.214540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.214956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.282049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adb3ec-980e-4a17-875a-bb67e78ddbed None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.290028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.291715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.319125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.320006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.320392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.320773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.328800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.331036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:10.343818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:10.345067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:10.346216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.352617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 406/812] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:10.352617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.440285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.467883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0b11-de90-42ac-899c-f67af67bb451 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.505737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.506356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.506646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.506930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.507623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:10.507935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.509351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.522750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 407/813] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:10.523641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.542126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 407/814] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:10.543681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.557797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.640637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fda8e7-f34f-4f07-adf9-9b383b5e59c8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.658890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2d545-4e22-49b6-93f9-b83416dfe23c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.685083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.685736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.686020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.686294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.687010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.688538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.691460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.697618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.698238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.698521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.698794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.699537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:10.699842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:10.700216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:10.701158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 16:05:10.704388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.705853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.705941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 408/815] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:10.705941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.719696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 408/816] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:10.725323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.807234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:10.817982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b52bca-d5b8-4448-89ca-9d047943f8b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.845524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd248d9-ffb1-46b5-90f0-4da87d67b9a0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:10.863308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.864021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.864307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.864583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.872938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.874385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:10.887025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:10.888877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 409/817] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:10.890083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.901472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:10.902118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:10.902401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:10.902680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:10.913470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:10.914930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:10.931157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:10.932375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:10.933479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:10.933479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:10.933479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:10.937007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 409/818] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:10.937845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:10.962672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.010691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e2e54-554d-4239-a58f-d358a4d8cb97 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.049746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50bb3-87be-43a5-b030-9035c3a9566a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.054242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.054856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.055138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.055443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.056082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] RBAC: Adding query filter params (name=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:11.056382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.058147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.062639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 410/819] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:11.063311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.076822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.093206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.093958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.094243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.094572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/058f62fc005a41b08ee30fb42192e31a/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.097146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=058f62fc005a41b08ee30fb42192e31a, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.098616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.123256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 410/820] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:10 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/058f62fc005a41b08ee30fb42192e31a/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:11.123256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.205212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3eea3-3f58-4c4d-8232-79e7a9bdcc6e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.232999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.248782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.249812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.250343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.250825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.252443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:11.252992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.254723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.259971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2134f-1e9a-4504-90f4-a7dffd03537a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.276822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 411/821] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:11.282585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.309710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.310842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.311125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.311430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.312376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:11.312685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.314246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.330151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 411/822] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:11.331542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.391192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.412496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc50e6-86a1-4dfd-a3f4-0b9381e1be66 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.456319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.456977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.457260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.457539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.458186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:11.458492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.459914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.470086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 412/823] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:11.470086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.486930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d52256-8bd6-464e-8c2d-60f95c2beb1a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.528009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.528734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.529117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.529553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.543038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.544723 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.548391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:11.564055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:11.565326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:11.566250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:11.566250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:11.566250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:11.571474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 412/824] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:11.572745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.629192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d085-2fd2-4abc-bdb1-cc5f9afd0223 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.673279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.675386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.675950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.676428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.686265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.688346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.694453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:11.705024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:11.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:11.706943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:11.707413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17e15b-f693-479f-ad7b-7e28f757a0a8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.714059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 413/825] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:11.714059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.754746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.755431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.755713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.755987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.756873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.758945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.791436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 413/826] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:11.792986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:11.895629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957c531-fc09-4522-b1d6-9fffd6d06e14 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:11.899186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:11.984553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:11.985148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:11.985430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:11.985706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:11.987472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:11.987584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:11.988910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:11.991268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1a6b1-a509-476b-adf1-13ba7b2f1f0e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.002467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 414/827] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:12.012187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.046023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.046563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.046848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.047126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.048212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:12.048532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.051183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd6bf-cd57-4e6b-a9f0-e51121afb1d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.066371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 414/828] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:12.067665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f21f52-2a62-4830-a730-546d5ab3a238 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.067940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f21f52-2a62-4830-a730-546d5ab3a238 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.068218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f21f52-2a62-4830-a730-546d5ab3a238 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.069525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 415/829] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:12.074055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f21f52-2a62-4830-a730-546d5ab3a238 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.087624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.151582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5e7d-0f51-49ae-8288-927817519bcb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.190340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.190971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.191280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.191739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.198070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f21f52-2a62-4830-a730-546d5ab3a238 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.199005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.201048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f25c04f1-c571-4e0d-a731-841f674ee1c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:12.212165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:12.213092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:12.214096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 415/830] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:12.215492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.215492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.215492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.232650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:12.234024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:12.239651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.240252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.240532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.240805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.241535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:12.241845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.243297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.259269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 416/831] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:12.259269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.260694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.335512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 416/832] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:12.364302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.401801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f6cc-06f1-4f6d-8a88-e4c0baf42ebe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.442531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.443132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.443436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.443707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.444395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:12.444762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.446186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.459460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 417/833] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:12.464965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.572295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a6e29-2d93-4fb6-86be-f526c47a7770 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.607586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.608164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.608446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.608716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/53087055f7e0417b99552e16781f4542/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.625397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=53087055f7e0417b99552e16781f4542, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.631255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.633647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.645195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 417/834] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:12 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/53087055f7e0417b99552e16781f4542/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:12.645406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.771708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea79854-a2e0-438d-b6e9-eca35840f992 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.805414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:12.811414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.812061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.812343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.812618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.813266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.814706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:12.827285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:12.828318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:12.829167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:12.830416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 418/835] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:12.836061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:12.941731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1699b6-049b-489c-a095-0a96a62f7abb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:12.981325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:12.981947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:12.982233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:12.982509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:12.983150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:12.983487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:12.984861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc94fa-d999-4f84-a85b-90bbd00b9d1e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:12.990454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 418/836] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:12.996742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.103199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8604-8991-4e8c-b042-43a8f01bbf3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.132669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9ea47-f16f-487c-a1ea-968ee63e1f01 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.132995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9ea47-f16f-487c-a1ea-968ee63e1f01 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.133226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9ea47-f16f-487c-a1ea-968ee63e1f01 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.134794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 419/837] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:13.139722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.139897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.140188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.140357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.140451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.140610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.140866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.141500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:13.141804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.143719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.149025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 419/838] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:13.160651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:13.161867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:13.228542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.238783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 420/839] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:13.509323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.573739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.624130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2c539-3cb1-4a58-8342-c542a7d8f5f0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.658269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.658897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.659178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.659482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.670059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:13.675796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:13.676745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:13.678110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 420/840] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:13.684572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.787175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564feaa-c679-4507-94a6-9b003e98d09f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.822785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.823812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.824360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.824840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:13.825716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:13.826223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:13.827832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:13.837570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 421/841] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:13.843042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:13.953992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8a5f-d47d-4879-acb1-92aa3c057897 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:13.977493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:13.994231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:13.994899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:13.995184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:13.995487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.007159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.008652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:14.023009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:14.024074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:14.025535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:14.025535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:14.025535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:14.025535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 421/842] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:14.029899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.087918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.136641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72731a-f990-430b-9c77-118718a21fa8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.172809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.173681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.174133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.174557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.175386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:14.175836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.177363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7be59-a3fe-43f9-905a-f8f3d25d75c0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.190834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 422/843] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:14.196425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.308855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7810bd-2f71-479f-b1a0-3c8664e3086a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.346634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.347570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.348003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.348425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.357803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.359540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.369863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3348-0d26-4943-9262-ec393318f53c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.370459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3348-0d26-4943-9262-ec393318f53c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.370949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3348-0d26-4943-9262-ec393318f53c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.372797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 423/844] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2000ffd9-5984-43b6-922c-c17cef132692 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:14.377394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:14.378341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:14.379322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 422/845] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:14.380968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea3348-0d26-4943-9262-ec393318f53c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.382063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.382354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.382633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.405920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:14.407109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:14.435240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.489306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 423/846] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:14.517307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea3348-0d26-4943-9262-ec393318f53c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.535373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.554273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.555087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.555539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.555986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.556856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:14.557308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.558933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2d3a-5dea-4f0a-9c70-71b0cc88e094 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.577677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 424/847] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:14.585580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.700715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bdcff-5505-4d14-b9f3-95a8ced0f4ea None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.737569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32b6ee-19a3-43ee-8d51-ee03bb133f07 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.737915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32b6ee-19a3-43ee-8d51-ee03bb133f07 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.738165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32b6ee-19a3-43ee-8d51-ee03bb133f07 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.739335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:14.739792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 425/848] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:14.739979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.740224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.740498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.744954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:14.745230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:14.745511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:14.768356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:14.769580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:14.774972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:14.821816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:14.823339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:14.851271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 426/849] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:14.851271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.947470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 424/850] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:14.956138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:14.989461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd0a5-8cf9-4e82-a839-cc0ff4851e9e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:14.991594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.025782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.026999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.027765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.028196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.028347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.028945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.030491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:15.041057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:15.042093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:15.043019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 427/851] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:15.048281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.073453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43fc6d-1527-4434-9c07-4327a8cacb4a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.112093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.113052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.113531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.113979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.126445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.128804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:15.145277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:15.146291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:15.147291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:15.147291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:15.147291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:15.147291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 425/852] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:15.147291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.168117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e07e45-ca55-474d-9c28-9f49dd304d4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.203857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.204382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.204830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.205099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.205741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:15.206040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.207430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.216733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 428/853] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:15.216733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.246580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.257040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224ed23-1c32-46c1-b831-b974f25258cb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.271172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.291802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.292425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.292701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.292970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.303173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:15.308510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:15.309605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:15.310619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 426/854] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:15.311409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.321291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc3ebd-c08d-43df-ba1c-e99658426926 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.356480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.357091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.357377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.357656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.358294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:15.358601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.359999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eab758-d2c3-4c9e-9ff0-b5c3c00a218a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.372035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 429/855] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:15.373210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c488a-4592-4080-9146-391206d60a01 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.373492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c488a-4592-4080-9146-391206d60a01 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.373761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c488a-4592-4080-9146-391206d60a01 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.379140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 430/856] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:15.379140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c488a-4592-4080-9146-391206d60a01 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.414912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b9e00-6ca6-4084-962a-c2b53eb8e27f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.448086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.448686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.448971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.449247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.449883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:15.450189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.451600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.457105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 427/857] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:15.457650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.471762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.481039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c488a-4592-4080-9146-391206d60a01 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.518040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.523139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.523139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.523434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.525442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.526252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:15.526559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.528075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e645b78-0cfd-4d97-9f3a-483af5ba8a76 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.538098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 431/858] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:15.539528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.539814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.540094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.559591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:15.560827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:15.599726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce919be-77c0-49b0-8101-f4c947c3c1d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.638170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 432/859] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:15.639049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.653562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.654948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.655464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.656176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.666566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.668792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:15.685320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:15.686427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:15.687505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:15.693701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 428/860] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:15.695515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.739679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.759206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f0f5-ccc0-4a97-8725-95f63798a969 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.805801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.807155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.807666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.808169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.821087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:15.823580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.824784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd826-9003-4559-8e4e-e59b7a51fabe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:15.825662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:15.846966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:15.848010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:15.848893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:15.848893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:15.848893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:15.853252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 433/861] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:15.854660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.867975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:15.869458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:15.869958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:15.870482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:15.871846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:15.872626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:15.874560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:15.897688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 429/862] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:15.899146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:15.983551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68cb5-719a-4079-99fd-e229b48524f5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.014514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4172-8901-4572-9100-b40f43455752 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.020357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.020972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.021258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.021535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.022177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:16.022490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.023889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.026404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.039303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 434/863] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:16.044263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:16.052464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.053071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.053353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.053626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.054348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=058f62fc005a41b08ee30fb42192e31a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:16.054657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.056137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.062510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 430/864] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:15 2022] GET /identity/v3/role_assignments?user.id=058f62fc005a41b08ee30fb42192e31a&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:16.063437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:16.097316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.175294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaf43-b6ff-4779-a438-d03c289a3346 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.196927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e056040-3a04-48c9-b558-f5a87f36b340 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.219397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.220388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.220915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.221351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.222217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.223977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.245483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 435/865] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:16 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:16.259479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.260554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.261031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.261491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.272860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.274736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:16.288653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:16.289829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:16.290983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:16.293110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 431/866] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:16.299736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:16.366860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.427323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe718-61e8-48df-a080-3e3a54f98db2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.463696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.464311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.464598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.464873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.465622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:16.465986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.467515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.489032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 436/867] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:16.496365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:16.584712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.615453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e843dc-55d7-4c6b-a931-7040dff615f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.652864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.653475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.653758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.654033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.725161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.726627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:16.806296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.829274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 432/868] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:16 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:16.835379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:16.946204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b4c07-353f-4319-b104-50e61789c037 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:16.961830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:16.983283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:16.984151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:16.984580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:16.985030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:16.996361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:16.997999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:17.013754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:17.014906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:17.016014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:17.016014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:17.016014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:17.017706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 437/869] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:17.027265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:17.132143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36631004-dd46-452d-b062-47b5f3a878e9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:17.167082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:17.167722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.168006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.168283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.168935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:17.169237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:17.170616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:17.183537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 433/870] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:17.188882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:17.294790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e495d1-2e43-4362-a706-b5f9030076a9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:17.331938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:17.332578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.332859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.333133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.340311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:17.341753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-124bfdee-8615-4339-9b8f-4846deb46eb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:17.353848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:17.354763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:17.355712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 438/871] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:17.360911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:17.405649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:17.432200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22a98-e70b-4031-a6fd-e0c465bed8e3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.432648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22a98-e70b-4031-a6fd-e0c465bed8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.433003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22a98-e70b-4031-a6fd-e0c465bed8e3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.434686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 439/872] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:17.440515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.440877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.441227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.467616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:17.468833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:17.486579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358eba-acd0-45b3-ac5d-57b8b6897848 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:17.495478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:17.529656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:17.530260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.530635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.530911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.531651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:17.531959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:17.533547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:17.547046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 434/873] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:17.553757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:17.563541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 440/874] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:17.661812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:17.666666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc42c-ea36-44aa-9114-94a40d5a6c55 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:17.704345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:17.705061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.705352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.705626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.706358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=53087055f7e0417b99552e16781f4542) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:17.706673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:17.708148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113ef6d-1006-40ee-9de7-7d8f39c8cff9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:17.714817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 435/875] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:17 2022] GET /identity/v3/role_assignments?user.id=53087055f7e0417b99552e16781f4542&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:17.838142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:17.952664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7150d8-c79b-4aa1-bd2f-f2d14e7968b8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:17.990936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:17.991585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:17.991871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:17.992148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:17.992811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:17.994211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a069f8-2557-42a3-9968-66a62825dd65 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:18.011578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 441/876] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:17 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:18.217221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c7c3d-df78-4173-885a-1474dc7920ef None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.217566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c7c3d-df78-4173-885a-1474dc7920ef None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.217672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c7c3d-df78-4173-885a-1474dc7920ef None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.219290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 436/877] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:18.225261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1594-22a1-48c7-870d-dd4906fd9b61 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.225553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1594-22a1-48c7-870d-dd4906fd9b61 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.225830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1594-22a1-48c7-870d-dd4906fd9b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.245195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-17ef1594-22a1-48c7-870d-dd4906fd9b61 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:18.246403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17ef1594-22a1-48c7-870d-dd4906fd9b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:18.258498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:18.327030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 442/878] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:18.375639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09499828-ab5d-42bf-9f03-59c32d88ea4e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.375948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09499828-ab5d-42bf-9f03-59c32d88ea4e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.376204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09499828-ab5d-42bf-9f03-59c32d88ea4e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.377883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 437/879] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:18.383350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.383654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.383938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.404271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:18.406488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:18.485781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 443/880] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:18.604878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09499828-ab5d-42bf-9f03-59c32d88ea4e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:18.711987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09499828-ab5d-42bf-9f03-59c32d88ea4e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:18.746978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:18.747554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.747833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.748106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.748692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:18.750056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:18.759840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:18.760892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.762406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 438/881] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:18.767070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:18.774156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:18.803422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:18.849374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:18.884100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d949fa-8ff2-4705-a2a0-c48d721cbe24 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:18.884336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2592071-e45c-47c2-bf67-aabc6742d710 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:18.922455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:18.923064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.923960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.924248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.924889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:18.925193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:18.925662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:18.926523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:18.926630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:18.927111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:18.927707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:18.928582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:18.930144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:18.932026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 444/882] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:18.937662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:18.942169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:18.943150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:18.944010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 439/883] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:18.952310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:19.052902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3002999-6f9d-41ef-aa19-fd30f1a5903c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:19.067615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:19.078468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a7a48-1096-44ee-b930-5e5021cd49fa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:19.091855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:19.092761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.093188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.093664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.094524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:19.095046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:19.096869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:19.099639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:19.107681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 445/884] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:19.122325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:19.122980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.123293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.123575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.124234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:19.124541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:19.125972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:19.136500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 440/885] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:19.138662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:19.259723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c95b-3a53-4f5c-a7c4-0bf733be3947 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:19.298420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:19.299013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.299326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.299607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.300240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:19.300547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:19.301918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:19.311399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 446/886] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:19.313324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:19.419525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb102c7-a480-45d1-960a-2e596b15a1d3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:19.458483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:19.459101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.459416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.459692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.460408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] RBAC: Adding query filter params (interface=public, service_id=99b5772c1c53481d88d12442838a2c1a, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:19.460717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:19.462100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162d3f2-e1a3-4f89-a3d5-f44814781e54 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:19.471333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 441/887] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:19 2022] GET /identity/v3/endpoints?service_id=99b5772c1c53481d88d12442838a2c1a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:19.473644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:19.541421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:19.582445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b3f8-2b79-4481-a515-cb8925502796 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:19.625806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:19.626440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.627266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.627549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.628177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:19.629566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2cf24-9185-40a2-a849-00640f863458 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:19.646115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 447/888] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:19 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:19.873072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97accb3-61bd-42da-9553-78a739e1d73c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.873566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97accb3-61bd-42da-9553-78a739e1d73c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.873566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97accb3-61bd-42da-9553-78a739e1d73c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.875012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 442/889] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:19.880232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef68f-d148-4b7c-9ddc-15d57c963fbf None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:19.880526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef68f-d148-4b7c-9ddc-15d57c963fbf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:19.880808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef68f-d148-4b7c-9ddc-15d57c963fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:19.901044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c0fef68f-d148-4b7c-9ddc-15d57c963fbf None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:19.902250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c0fef68f-d148-4b7c-9ddc-15d57c963fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:19.971351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 448/890] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:20.049037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26532d4-e896-4df7-8bb3-9c0170dd4bbf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.049394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26532d4-e896-4df7-8bb3-9c0170dd4bbf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.049607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26532d4-e896-4df7-8bb3-9c0170dd4bbf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.051109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 443/891] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:20.057368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.057882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.057961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.078681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:20.080472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:20.136742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.152528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 449/892] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:20.264222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26532d4-e896-4df7-8bb3-9c0170dd4bbf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.285687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.377178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26532d4-e896-4df7-8bb3-9c0170dd4bbf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.415330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.415958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.416239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.416514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.427513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.428319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:20.434210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:20.435129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:20.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 444/893] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:20.441510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.534214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b1a2-8419-4168-9b5d-48278da13fa5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.549064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.572335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.573311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.573809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.574302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.575235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:20.576817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:20.578564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca433474-75b5-4a3a-b728-baf748788c61 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:20.588055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:20.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:20.589914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:20.597548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 450/894] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:20.598778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.629404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.630044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.630333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.630608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.631322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:20.631789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:20.633486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:20.647094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 445/895] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:20.652308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.669472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.734924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0f2fb-0e82-49f8-891b-350d70ddf69b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.780872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.781751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.782038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.782317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.783767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:20.784087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:20.785493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:20.792789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 451/896] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:20.793360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7ee32-e0ce-4763-9a70-13cdace67529 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.800582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.842563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.843208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.843519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.843793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.856653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.861758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:20.863335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:20.883986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:20.885405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:20.886300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:20.886300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:20.886300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:20.886300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 446/897] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:20.900573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:20.919577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65431d-eb52-4300-9400-13fc2df08d03 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:20.938410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:20.959079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:20.960035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:20.960491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:20.960950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:20.961757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:20.962310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:20.964104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:20.974858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 452/898] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:20.977008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.030388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f671b96-8ccf-4c87-99b8-595f3ebe21c6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.077471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.078205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.078552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.078958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.081091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:21.081590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.084514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.097493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21834c91-0303-4884-842d-eb2dfc7c8967 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.107385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 447/899] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:21.111671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.138650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.139522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.140097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.140524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.141430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] RBAC: Adding query filter params (interface=public, service_id=972896b177994042b00d59d493509b7d, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:21.141887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.144176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2638192-b0cf-4936-ba19-203bf90fda1e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.149845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 453/900] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:20 2022] GET /identity/v3/endpoints?service_id=972896b177994042b00d59d493509b7d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:21.151109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c0a0-ca27-4241-bd92-7da208b76c18 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.151434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c0a0-ca27-4241-bd92-7da208b76c18 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.151716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c0a0-ca27-4241-bd92-7da208b76c18 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.153178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 454/901] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:21.158401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221c0a0-ca27-4241-bd92-7da208b76c18 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.171436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:21.213074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:21.224334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c61a9-1766-466f-aa8e-731526647299 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.259505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.260124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.260402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.260672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.268684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.270125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.280446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221c0a0-ca27-4241-bd92-7da208b76c18 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2cf345fc-e569-436b-a87d-3d9d962dc85b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:21.282116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:21.284677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 448/902] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.285660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.286372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.305399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:21.306584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:21.324546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.325440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.325727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.326003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.326613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.328471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.335709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 455/903] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:21 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:21.336532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.397063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 449/904] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:21.465347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e14e1e-aa95-419a-9404-74e2811790c5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.476625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:21.505836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.506463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.506744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.507020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.507796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:21.508103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.509549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.529683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 456/905] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:21.535739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.587666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:21.644508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96ace8-a9b9-4a70-b7de-d93bd98661c7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.670438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.682662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.683416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.683702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.683975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.765284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.767239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.802309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79af9b-d63d-42fb-8045-43707f89b18b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:21.814938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:21.842530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:21.843146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.843458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.843734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.844406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:21.846338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef418a6-0c47-4378-acd5-383171e2d152 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:21.863309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 457/906] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:21 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:21.864901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d06b-52d7-4946-8111-d35fc775b1bb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.865401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d06b-52d7-4946-8111-d35fc775b1bb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.865837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d06b-52d7-4946-8111-d35fc775b1bb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.867627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 458/907] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:21.883474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:21.883474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:21.883474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:21.897200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 450/908] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:21 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:21.902579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:21.903029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:21.904434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:21.951307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.013099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 459/909] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:22.028384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf06e-1eeb-40e1-aa4b-2e346ae755cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.071612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.072604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.073161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.073635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.085374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.087129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.089186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:22.103737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:22.104721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:22.105620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:22.105620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:22.105620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:22.107310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 451/910] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:22.112652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.218982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41cf6b-06a6-4924-9f08-703b2547d728 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.255537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.256152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.256433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.256712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.257361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:22.257670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.259045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.272358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 460/911] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:22.277891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.289289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.305951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.395275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588bd56-6c22-4a6f-bde5-ebfa4d7d82db None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.421615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77626c34-9239-427d-a6c9-f4b8bec23f93 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.438364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.439738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.440028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.440750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.449859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.452148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.462439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.463114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.463416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.463701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.464296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.465674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:22.466464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:22.467331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:22.468233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:22.471315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 452/912] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:22.473662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:22.480852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:22.481884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 16:05:22.482715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:22.487313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 461/913] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:22.488545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.549466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.594253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.598666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b5b3c-0440-4173-93be-4720c83fb987 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.625512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a389-5645-4eed-b9e1-2e2b69cef226 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.659609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.660521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.661240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.661737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.662761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:22.663304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.665210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.677316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.677925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.678210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.678490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.679122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:22.679461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.680821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.680939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 453/914] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:22.687905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.695275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 462/915] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:22.697907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:22.819924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3681-d111-4581-abcf-092aeabc3f16 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.820939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4499c1-673b-4686-a282-cf2ab5b0e43b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:22.836562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:22.859068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.859722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.860019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.860290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.861044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] RBAC: Adding query filter params (role.id=9d8dcbdd28914d56b935a45e3063c79b, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=53087055f7e0417b99552e16781f4542) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:22.861610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.863059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.864902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:22.865888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:22.866380 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:22.866907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:22.867814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:22.868355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:22.870001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05549-099c-4a04-897c-18d0d01787cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:22.880794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 454/917] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/role_assignments?user.id=53087055f7e0417b99552e16781f4542&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=9d8dcbdd28914d56b935a45e3063c79b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:22.881245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 463/917] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 30 16:05:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:22.882893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:23.008003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cb7e-81d9-47fd-8c01-d070be87be34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:23.044912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:23.045513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.045791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.046062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.046752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:23.048148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed78b75-04c3-421b-82b7-1ca863834e6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:23.070856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 455/918] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:23.395738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5940-1a3d-47e6-8e1e-46ff6d7dfbee None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.396262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5940-1a3d-47e6-8e1e-46ff6d7dfbee None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.396262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5940-1a3d-47e6-8e1e-46ff6d7dfbee None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.397782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 464/919] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:23.403267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36bb52-5553-4e38-b398-212184e0c491 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.403566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36bb52-5553-4e38-b398-212184e0c491 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.403848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36bb52-5553-4e38-b398-212184e0c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.423864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5b36bb52-5553-4e38-b398-212184e0c491 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:23.425058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b36bb52-5553-4e38-b398-212184e0c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:23.452300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:23.501557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 456/920] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:23.773397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5940-1a3d-47e6-8e1e-46ff6d7dfbee None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:23.778409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:23.899034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5940-1a3d-47e6-8e1e-46ff6d7dfbee None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:23.924925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7294-82cf-461e-a5f6-8aa4086a2a98 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.925268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7294-82cf-461e-a5f6-8aa4086a2a98 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.925467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d7294-82cf-461e-a5f6-8aa4086a2a98 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.927079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 457/921] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:23.932130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.932414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.932693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.936436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:23.937062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:23.937441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:23.937719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:23.938301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:23.939815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:23.948836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:23.950065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:23.951480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 465/922] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 30 16:05:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:23.956020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:23.956214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:23.957505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:24.042128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.050071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 458/923] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:24.083477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2127-74bc-40b7-968f-8a52ce1cd61d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.120427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.121315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.121754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.122184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.122966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:24.123534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.125049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.130513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 466/924] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:24.136538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.246460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.259322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40d3f6-f9c0-46c7-b6ea-dcac485e5c57 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.297641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.298605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.299045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.299511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:24.300769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.302299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.308040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 459/925] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:24.313975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.315764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.426924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40809a-0007-4f58-abb8-430a07d36d10 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.442156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612b72a-1583-4d99-bb4b-bdc884a5ce9a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.463919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.464687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.465088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.465462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.466136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.467644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:24.476882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:24.478092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:24.479135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:24.485613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 460/926] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:24.487359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.488157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.493131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.494088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.494573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.495070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.495978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.498188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.508900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.531136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 467/927] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:24.621377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bad9f-4307-4be8-bdcd-dbde91eb2589 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.669760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.670375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.670663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.670936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.671601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:24.671905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.673490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.679210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 461/928] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:24.684912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.797707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379d9d-a4bb-4fb4-8caf-f291bd932404 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:24.833254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:24.833881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.834163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.834441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.835080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:24.835411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:24.836781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b34e1a-d208-4393-b671-cb5f1ad62e7a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:24.842271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 468/929] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:24.849720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:24.857728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d618-a37f-4c3f-a1bd-c604cda9d9d7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.858211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d618-a37f-4c3f-a1bd-c604cda9d9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.864803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d618-a37f-4c3f-a1bd-c604cda9d9d7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.864803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 469/930] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:24.868555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:24.868843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:24.869123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:24.887543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:24.888747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:24.948006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.952370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:24.972395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 470/931] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:24.989928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27b7ed-0ac5-4573-a6dc-382b0434ce13 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.025933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.026965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.027471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.027963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.028876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] RBAC: Adding query filter params (interface=public, service_id=4fb602066f684e908b4a4b096cf462c2, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:25.029381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.031006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f71a-c30b-4a89-a438-e523c516e195 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.037068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 462/932] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:24 2022] GET /identity/v3/endpoints?service_id=4fb602066f684e908b4a4b096cf462c2&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:25.043252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.151056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21f8ee-4912-42ec-8f6e-eda5d005bb2a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.163444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150225dc-702f-40a1-8d11-421f411f8bb3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.163965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150225dc-702f-40a1-8d11-421f411f8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.163965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150225dc-702f-40a1-8d11-421f411f8bb3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.165550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 463/933] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:25.171825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.172354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.173089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.195577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.196625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.196745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:25.197273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.197773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.198448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:25.198820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.200436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.208554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 471/934] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:25.269510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.279271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 464/935] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:25.330325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:25.390625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cde55-b056-4fa0-ba24-3d993287c045 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.427256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.428156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.428579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.429007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.440612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:25.445547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:25.447124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:25.448201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.450487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 472/936] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:25.459260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.534281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:25.566106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859b4fa-3af5-4622-82e3-79deecbcb399 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.570691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.603688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.604270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.604618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.605104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.606358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:25.606910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.608546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.628643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 465/937] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:25.630073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.687318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ff067-de43-4d69-a167-a48bcb3dbee7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.696022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:25.728321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.729141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.729559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.729929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.730685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.732538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.735361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING keystone.server.flask.common [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 16:05:25.752789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d19f-190e-49a6-bf2e-617ad686ec64 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.756977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.sql.core [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 16:05:25.756977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 16:05:25.756977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [parameters: {'id': 'b6b6bea9785c481bb7e3d9ff1973ac6b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 16:05:25.756977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81606) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b6b6bea9785c481bb7e3d9ff1973ac6b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b6b6bea9785c481bb7e3d9ff1973ac6b. Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:05:25.765236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 16:05:25.766054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application yield resource Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise self.value Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application session.commit() Aug 30 16:05:25.767054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.flush() Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application compat.raise_( Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 16:05:25.768395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 16:05:25.769732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:05:25.771248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [parameters: {'id': 'b6b6bea9785c481bb7e3d9ff1973ac6b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.773128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:25.774880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b6b6bea9785c481bb7e3d9ff1973ac6b. Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 473/938] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Aug 30 16:05:25 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:25.776779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.803432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.804441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.804910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.805419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.807564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:25.818457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.820215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:25.835747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:25.836783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:25.837729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:25.837729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:25.837729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:25.843371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 466/939] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:25.845909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.896239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d9e34-1f41-420e-8ba0-ea42ab54d030 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.937910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.938705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.939181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.940319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.948543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:25.950658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:25.956569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941894-f475-47f3-9905-29cefd7e6463 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:25.963082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:25.964215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:25.965313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:25.971491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 474/940] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:25.972724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:25.996508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:25.997428 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:25.997875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:25.998308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:25.999179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:25.999677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.000816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:26.001453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.019306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 467/941] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:26.020783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.070097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:26.093877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efe86c-63ad-409d-b042-e6232120448e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.128102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3a77-6ccb-42ab-bd82-2882b5cf6103 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.135240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.135994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.136372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.136747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.137609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:26.138007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.139853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.166274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.167711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.168221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.168731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.169311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 475/942] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:26.178518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.180487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:26.195834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:26.196823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:26.197809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:26.204708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 468/943] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:26.207045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.301009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:26.330036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22ed78-092c-496c-b36b-b7120ffa06f1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.367489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.368347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.368625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.368896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.369618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:26.369919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.371352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5f8ad-5973-4613-959f-fea3db616332 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.390509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 476/944] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:26.397265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.510812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0c569-5671-43ac-8fa4-99e7b690d77e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.518495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:26.529806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de761-4ac1-42ea-8d89-711924adde41 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.530328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de761-4ac1-42ea-8d89-711924adde41 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.530328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de761-4ac1-42ea-8d89-711924adde41 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.531963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 477/945] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:26.539240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.539542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.539822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.554876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.555887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.556407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.556893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.560545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:26.562083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:26.570000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.571936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:26.587849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:26.588840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:26.589722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:26.589722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:26.589722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:26.592016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 469/946] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:26.598626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.663258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 478/947] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:26.724031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82918-d6f6-4ccd-bda3-21a1a50f1081 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.762425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.762957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.763264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.763549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.764209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:26.764519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.766048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.786765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 470/948] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:26.792464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.867744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:26.902590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d20cdb-4ea3-4a99-964a-585e20d5ab33 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:26.937998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:26.938637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:26.939464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:26.939742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:26.946817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:26.948355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:26.960376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:26.961263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:26.962159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:26.966379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 479/949] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:26.967933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:26.996098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:27.014077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:27.090159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a99d5-ed06-46e7-b7fa-f7656cf049ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.129482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.130415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.130892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.131389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.132161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4361d2-c4a4-4ed9-a2c2-cd958e1eac78 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.132494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:27.132979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.134825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.152901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 471/950] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:27.154568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:27.164309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:27.171209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.172184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.172678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.173723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.174457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.176015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5aaacd84-bf39-4cdc-8ea9-99d4829bbabe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:27.185213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:27.186406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:27.187481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:27.189715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 480/951] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:27.191156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf64ab-95a9-4b6e-bb4f-34686187f32c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.191648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf64ab-95a9-4b6e-bb4f-34686187f32c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.192194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf64ab-95a9-4b6e-bb4f-34686187f32c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.194128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 481/952] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:27.195550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf64ab-95a9-4b6e-bb4f-34686187f32c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:27.248217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:27.271180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e5bf-bbd3-4f09-a87e-000d08ae045d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.308711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf64ab-95a9-4b6e-bb4f-34686187f32c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.313104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.314416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.314756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.315032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/53087055f7e0417b99552e16781f4542/roles/9d8dcbdd28914d56b935a45e3063c79b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.317199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=53087055f7e0417b99552e16781f4542, role_id=9d8dcbdd28914d56b935a45e3063c79b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.318630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae97857-ca5b-450a-923f-f1b5a00eeeed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.333548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 472/953] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:27 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/53087055f7e0417b99552e16781f4542/roles/9d8dcbdd28914d56b935a45e3063c79b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:27.335088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.335423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.335700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.355213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:27.356546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:27.365003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.366817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.367395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.367941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.369389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:27.369978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.372393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.380968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 482/954] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:27.386722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:27.453492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 473/955] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:27.465477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:27.518106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01ea4a-8739-4684-9bd9-e12dcc5c876b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.555215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.555862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.556073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.556293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.556915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:27.557212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.558566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.563388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 483/956] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:27.728289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:27.797799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:27.838568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8536-c056-4485-a8b2-db0d4aa5fbd8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:27.875361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:27.875997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:27.876804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:27.877090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:27.877677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:27.879042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:27.889583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:27.890766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:27.892703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 474/957] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:27.897385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:28.002636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646228aa-c076-4112-9018-5b5a6af56f07 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:28.027680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:28.041391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:28.042005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.042283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.042557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.043195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:28.043528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:28.044963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:28.051479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 484/958] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:28.058546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:28.166775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22875-c940-47a8-9b9d-ab36e5621596 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:28.196921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:28.202589 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:28.203207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.203521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.204174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.205986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:28.206812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:28.208574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b30e8-72c1-4d08-9e1f-831da7deb3e2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:28.214653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 475/959] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:28.220141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:28.277622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466b94-5aff-4d09-89f7-d87b9635bcf1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.278141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466b94-5aff-4d09-89f7-d87b9635bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.278141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466b94-5aff-4d09-89f7-d87b9635bcf1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.279783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 476/960] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:28.284833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.285115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.285388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.303378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:28.304730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:28.333289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb017979-d11d-474f-a36c-74eed2a1e36d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:28.372088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:28.372700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.373084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.373365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.374045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:28.375438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:28.387440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 477/961] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:28.399247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 485/962] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:28.661673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:28.769091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76320-85a9-4696-b542-35998fd5f5bf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:28.795209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:28.809789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:28.810398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:28.810671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:28.810938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:28.811610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:28.813033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:28.815607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING keystone.server.flask.common [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 16:05:28.830758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 16:05:28.830758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 16:05:28.830758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [parameters: {'user_id': '392dc5dbaa9f47749a70b54601008028', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 16:05:28.830758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Aug 30 16:05:28.847527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:05:28.848460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 16:05:28.849430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 16:05:28.850461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 16:05:28.852287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 16:05:28.854020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'user_id': '392dc5dbaa9f47749a70b54601008028', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:28.855054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 16:05:28.856116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:28.857279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 478/963] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 30 16:05:28 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:28.857767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:28.911742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:28.972410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660d354-4fd4-4c23-b585-b383657af916 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:29.008380 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:29.009018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.009299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.009573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.018177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:29.019651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-318f2520-07dd-443c-ba57-2ef3aa3a85ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:29.031005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:29.033725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:29.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 486/964] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:29.043252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:29.150443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02adc10-9ac1-4f37-8063-a281f0ea5ee9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:29.187361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:29.188273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.188770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.189225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.190156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:29.190637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:29.192491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b882424-80bf-4c31-88a3-c21f28f0d138 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:29.234021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 479/965] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 30 16:05:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:29.484260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0354-d0ef-4842-b939-cbf4ab7b0e2d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.484606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0354-d0ef-4842-b939-cbf4ab7b0e2d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.484789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d0354-d0ef-4842-b939-cbf4ab7b0e2d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.486549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 487/966] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:29.492468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.492950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.493049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.512660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:29.514186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:29.555795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:29.588201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c83cd-76e3-45c5-b9c8-9d45fbe4868d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.588584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c83cd-76e3-45c5-b9c8-9d45fbe4868d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.588937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c83cd-76e3-45c5-b9c8-9d45fbe4868d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.590465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 488/967] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:29.591604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 480/968] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:29.596236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:29.596527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:29.596811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:29.615240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:29.616490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:29.695369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 489/969] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:29.867666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:29.962613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:29.984008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5529-e342-4274-b870-29154c79dbd5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:29.997585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.021838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.022405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.022689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.022971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.034192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:30.039001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:30.039954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:30.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 481/970] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:30.046012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.079096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.091976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea5cf9-1325-491d-bf73-a1127fd662b1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.131643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.132161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.132438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.132712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.133372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.134730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.155273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 490/971] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:29 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:30.163487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724ad4-ffc8-4e2e-a85b-180713c0c478 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.211706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.212327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.212602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.212869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.213542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:30.213846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.215252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170009fd-eeca-495c-8bbe-c24ac5478abd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.229496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 482/972] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:30.231255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.334591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.344422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7dfb4-f91c-4e4e-bf45-1830b69a56a3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.382315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.382863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.383142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.383454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.394956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.396440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:30.411654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:30.412715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:30.413623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:30.413623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:30.413623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:30.415827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64c7f-9c2f-49db-b3dc-98de6f051dd7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.416115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64c7f-9c2f-49db-b3dc-98de6f051dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.416391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64c7f-9c2f-49db-b3dc-98de6f051dd7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.417621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 491/973] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:30.417951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 483/974] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:30.418921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.423954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.424331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.424675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.444093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:30.445369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:30.522193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 484/975] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:30.531268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31120904-5a07-4bf3-9ca3-655addb4d8b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.567778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.568378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.568655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.568926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.569556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:30.569857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.571200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.583663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 492/976] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:30.589911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.616283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.708117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bde80-069e-477c-bcf2-3d423911ffa6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.744280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.744906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.745189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.745472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.753588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.755065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:30.768418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:30.769440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:30.770385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 485/977] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:30.775130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.778960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.796849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.886421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:30.900061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977fcf-6022-4be2-897b-5e7ca49eb261 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.911754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3978f-fdb8-43a9-9948-34120ada9499 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:30.936375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.936982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.937259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.937537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.938260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:30.938567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.940104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.948340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:30.948975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:30.949257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:30.949528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:30.950313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:30.951897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:30.959383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 493/978] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:30.962911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:30.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:30.964758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:30.966546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:30.967393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 486/979] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:30.971395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.035324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.078179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ffc2-e907-48be-ade6-9ebf20aa55ea None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.082200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b1581-3e4c-44a7-824a-2e32f79ba42e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.115900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.116548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.116829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.117106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.125176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.125902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.131029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.131029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.131029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:31.131029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.131029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.136915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 487/980] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:31.143239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.156266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.195504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.196964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.260566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018a4e-07b0-4a35-99d9-f81aa02316b1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.262512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.300030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.300642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.300927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.301205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.301846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:31.302149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.303567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.307170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 494/981] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:30 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:31.312729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.314277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 488/982] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:31.315145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.388716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.421016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766437d-a049-44c4-887c-4ae1c9087256 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.421357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c667f-6741-4bb3-ab22-6004cc6db164 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.455187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.455812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.455933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.456115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.456395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.456516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.456740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.456951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.457663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] RBAC: Adding query filter params (interface=admin, service_id=4fb602066f684e908b4a4b096cf462c2, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:31.457975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.459376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.464952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 489/983] 10.222.0.57 () {56 vars in 1249 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/endpoints?service_id=4fb602066f684e908b4a4b096cf462c2&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:31.466811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.468354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.470899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:31.488192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:31.489210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:31.490094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:31.490094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:31.490094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:31.490094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 495/984] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:31.495021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.510858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.577715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb2a5-3984-4146-9ebd-0e7324295935 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.599426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2638c-a09c-402a-91e4-d29d8c779975 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.614419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.615024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.615333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.615616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.616222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.617588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.619425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.630914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 490/985] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:31.647633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.648271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.648557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.648839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.649508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:31.649819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.651293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.669707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 496/986] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:31.671192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.782379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddfe05-a939-45ff-be97-f54447c14bea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.819401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.820032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.820307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.820579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.827572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:31.829002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:31.840999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:31.841921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:31.843129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 491/987] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:31.848509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:31.930418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:31.958164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7445-c39b-4a7a-ad30-090769be962a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:31.996315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:31.996950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:31.997197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:31.997475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:31.998177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:31.998486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:32.000036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8baf9-0ec1-4287-b32d-0495f2b30519 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:32.012302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 497/988] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:32.019020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:32.093237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:32.136922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047175-fb81-4086-93d8-48b3aa1131c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:32.152268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376db5-18ea-4484-a791-766bc39e015a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.152575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376db5-18ea-4484-a791-766bc39e015a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.152835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376db5-18ea-4484-a791-766bc39e015a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.154318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 498/989] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:32.159106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.159423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.159708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.174897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:32.175619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.176425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.176772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.177591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] RBAC: Adding query filter params (role.id=9d8dcbdd28914d56b935a45e3063c79b, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=53087055f7e0417b99552e16781f4542) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:32.177970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:32.179504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:32.181360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:32.182491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:32.191252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 492/990] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:32 2022] GET /identity/v3/role_assignments?user.id=53087055f7e0417b99552e16781f4542&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=9d8dcbdd28914d56b935a45e3063c79b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:32.258379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 499/991] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:32.522881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:32.630119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bd1-374c-411b-9b51-4260c52264f1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:32.659237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:32.671669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:32.672305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.672586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.672858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.673477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:32.674852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:32.683880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:32.684842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:32.685674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:32.687062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 493/992] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:32.690974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:32.757769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:32.799816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667701f8-784b-4dbc-abc9-75563dbb02ad None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:32.834131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:32.834781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.835063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.835426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.836052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:32.836352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:32.837700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:32.846930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 500/993] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:32.848509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:32.962975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ee574-32f5-4e3f-9e30-b677a82e4df1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:32.997926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:32.998525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:32.998800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:32.999068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:32.999725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:33.000095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:33.001453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:33.011244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 494/994] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:33.012516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:33.123925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c457c-8c24-4000-991e-37cd61855433 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:33.151825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:33.161096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:33.161711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.161996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.162299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.163007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] RBAC: Adding query filter params (interface=public, service_id=32255de61db54b968621aec4e3ec0d6e, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:33.163342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:33.164709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db145a2-d07a-4bb2-8347-37117e326195 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:33.171589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 501/995] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:33 2022] GET /identity/v3/endpoints?service_id=32255de61db54b968621aec4e3ec0d6e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:33.176248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:33.242606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:33.292464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545e596-4a0b-432a-9f2b-da5443c75d4e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:33.330390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:33.331001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.331322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.331598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.332207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:33.333577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81bbf4-08f2-4404-a401-8c8c4cdfc7a5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:33.347496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 495/996] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:33 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:05:33.561464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e18a5-a873-4fbe-836a-407d5bbe527d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.561802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e18a5-a873-4fbe-836a-407d5bbe527d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.562023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e18a5-a873-4fbe-836a-407d5bbe527d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.563818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 502/997] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:33.569418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:33.569693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:33.569968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:33.589385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:33.590558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:33.666035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 496/998] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:33.934403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e18a5-a873-4fbe-836a-407d5bbe527d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.047625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e18a5-a873-4fbe-836a-407d5bbe527d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.082991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.083648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.083930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.084202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.084782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.086154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6c463e32-b8da-4cda-8751-3ad733f11451 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:34.094929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:34.096161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:34.097680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 503/999] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 30 16:05:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:34.103284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.152646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:34.171985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca249f6-ffe0-461c-bd09-4dc716882cb3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.172313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca249f6-ffe0-461c-bd09-4dc716882cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.172804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca249f6-ffe0-461c-bd09-4dc716882cb3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.175286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 504/1000] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:34.181131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.181348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.181565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.204358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:34.205643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:34.210875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd419fd-671c-454a-bf8c-1eddd2232bbc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.247586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.248188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.248465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.248737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.249370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:34.249674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.251676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.256739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 497/1001] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:34.262759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.266069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:34.284162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 505/1002] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:34.373083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35115e-60e0-4615-8e6e-6c576249b4c1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.384734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:34.410071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.410677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.410958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.411254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.411921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:34.412229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.413606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.423522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 498/1003] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 30 16:05:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:34.424670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.531316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9a89-0609-4a75-8d11-93b28986c1f5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.556491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.568414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.568822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.569107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.569387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.570057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.571501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.595263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 506/1004] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:34.669878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5134-509a-4cae-8b75-654f27604f83 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.714125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.714770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.715054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.715357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.715952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.717364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:34.726999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:34.728237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:34.729258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:34.731400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 499/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 30 16:05:34 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:34.734154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.787369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:34.847400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44662d-3a9e-439a-b2ba-4a25a41c1a69 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:34.888611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:34.889218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:34.889498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:34.889827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:34.890463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] RBAC: Adding query filter params (name=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:34.890847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:34.892702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b3699-fea6-446f-8730-1d3545eeef35 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:34.898148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 507/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:34.903759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:34.947540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:35.016412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc063040-8103-4648-9ced-343adbbc78d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:35.050589 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:35.051129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:35.051403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:35.051678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:35.052303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:35.052605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:35.053981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9ddf5-974c-4dae-9a8a-2c6722667242 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:35.058958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 500/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:35.331256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04be46-8936-45a4-a5c5-1dd93205e310 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:35.331771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04be46-8936-45a4-a5c5-1dd93205e310 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:35.331771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04be46-8936-45a4-a5c5-1dd93205e310 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:35.333210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 508/1008] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:35.338203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:35.338496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:35.338772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:35.357571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:35.358750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:35.429438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 501/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:35.709704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04be46-8936-45a4-a5c5-1dd93205e310 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:35.819906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04be46-8936-45a4-a5c5-1dd93205e310 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:35.830902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:35.859876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:35.860952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:35.861683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:35.862158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:35.862970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:35.864407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:35.874816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:35.879553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:35.880497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 509/1010] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:35.882871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:35.913917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:35.995047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991d64-7719-4541-a2f1-1d20fcddb7c0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:36.031269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:36.031889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.032173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.032479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.033116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:36.033423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:36.034790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:36.044533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 502/1011] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:36.045993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:36.159377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1858d-5b75-46fc-84fc-0d40a48976a1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:36.195828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:36.196332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.196610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.196882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.197503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:36.197803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:36.199153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec2ad3-3914-47ef-9229-f21e31696b55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:36.209233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 510/1012] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 30 16:05:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:36.210856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:36.309467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:36.319334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de97ba-90bf-449e-b853-3c9d8430a4f8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:36.353801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:36.354414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.354696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.354969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.355681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:36.357061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead7cec-24d3-4c26-9d0d-6c9ff6319916 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:36.379363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 503/1013] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:36.610045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fba7e9-b7b3-4cdb-8c0e-fcaa8ac568d1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.610543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fba7e9-b7b3-4cdb-8c0e-fcaa8ac568d1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.610543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fba7e9-b7b3-4cdb-8c0e-fcaa8ac568d1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.612371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 511/1014] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:36.617531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad944e0-d347-4abf-813a-34ec9f99d46e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.617816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad944e0-d347-4abf-813a-34ec9f99d46e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.618086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad944e0-d347-4abf-813a-34ec9f99d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.639102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0ad944e0-d347-4abf-813a-34ec9f99d46e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:36.640370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0ad944e0-d347-4abf-813a-34ec9f99d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:36.716982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 504/1015] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:36 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:36.987385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fba7e9-b7b3-4cdb-8c0e-fcaa8ac568d1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:36.988507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb6713-50f7-46b7-8f72-df96b8689ad2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.988990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb6713-50f7-46b7-8f72-df96b8689ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.989070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb6713-50f7-46b7-8f72-df96b8689ad2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:36.990436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 505/1016] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:36.995606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:36.995895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:36.996175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:37.014801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:37.016030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:37.031886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:37.087054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 506/1017] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:36 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:37.102278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fba7e9-b7b3-4cdb-8c0e-fcaa8ac568d1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:37.138189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:37.138808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:37.139084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:37.139375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:37.140470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:37.142405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:37.144144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 512/1018] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:37.149678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:37.154167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:37.256377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644e1c0-b175-4ae6-832e-4d846a755759 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:37.292060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:37.292722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:37.293004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:37.293279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:37.293978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:37.295449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987f18d-cb0a-4445-84ee-779544766f3c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:37.323395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 507/1019] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:37 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:37.353072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:37.467269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3256-4d79-48b5-997a-4b868d1663d0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:37.474489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:37.504292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:37.504907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:37.505190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:37.505458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:37.506118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:37.507567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825beec-f5c1-417b-bbce-8954cff42285 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:37.524409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 513/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:05:37 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:38.432108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6960ef-c9bf-4bca-84fb-f69ad3a5dac7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:38.432633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6960ef-c9bf-4bca-84fb-f69ad3a5dac7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:38.432712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6960ef-c9bf-4bca-84fb-f69ad3a5dac7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:38.434041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 508/1021] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:38.439718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:38.440127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:38.440294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:38.458509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:38.459646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:38.537851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 514/1022] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:38.804571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6960ef-c9bf-4bca-84fb-f69ad3a5dac7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:38.862932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:38.914849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6960ef-c9bf-4bca-84fb-f69ad3a5dac7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:38.949258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:38.949889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:38.950169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:38.950445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:38.961807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:38.966555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:38.967700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:38.968613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 509/1023] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:38.975405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.070034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.082594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb426-aefd-4c2f-bdb8-6b77a82e6a68 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.117491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.118100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.118383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.118660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.119387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:39.119699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.121069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.134524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 515/1024] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:39.136000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.239802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d8a97-3362-46d2-990a-1670eb4b6339 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.256519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.275456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.276081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.276359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.276629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.287456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.288900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f64ac399-490f-4022-b237-11c7a280be7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:39.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:39.304439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:39.305385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:39.305385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:39.305385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:39.305385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 510/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:39.311676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.327015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af349-d42f-44ff-b258-1b4c32836ef1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.327263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af349-d42f-44ff-b258-1b4c32836ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.327525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af349-d42f-44ff-b258-1b4c32836ef1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.329543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 511/1026] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:39.334581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.334867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.335319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.360105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:39.362188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:39.435683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 512/1027] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:39.439190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c2074-baff-48ba-ad76-78fd3651db8b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.474037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.474696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.474914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.475185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.475862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:39.476164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.477531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6294037-51d3-48f2-8be8-dae5546fccd4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.490675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 516/1028] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:39.501653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.506961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038718-256d-4f15-9849-1a1012d9040c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.507275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038718-256d-4f15-9849-1a1012d9040c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.507552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038718-256d-4f15-9849-1a1012d9040c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.508921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 517/1029] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:39.514454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.514742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.515016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.534785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:39.536101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:39.546914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.600496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.605292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673b934-ea73-4633-81f9-408403b1bd20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.611417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 518/1030] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:39.650492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.651194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.651593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.651942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.659422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.660947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:39.672914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:39.673909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:39.674932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:39.678825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 513/1031] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:39.680136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.716640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.793496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77467-ee2d-44e7-b759-7a52f03cf82f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.824163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e05d-21dd-4f8e-ba74-246296d61f88 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.830793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.831434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.831720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.832225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.832727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:39.833035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.834461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.852577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 519/1032] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:39.857978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.860472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:39.861104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:39.861491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.861649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:39.861927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:39.863072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:39.864565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:39.866154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 514/1033] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:39.871769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:39.902991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:39.964939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f9e1-ded4-4290-a1c6-38db5b599c5a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.977207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e54a8-3251-4fc1-856d-6a04804893c2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:39.999839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.000539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.001532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.001760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.012368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.012967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.013251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.013519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.014159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.015624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.036936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 515/1034] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:05:39 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:40.037750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.077199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.078681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.081321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.139310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.162496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702001c0-355d-4675-92b9-f276a40646e9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.201612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 520/1035] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:05:40.206800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.207595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.207881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.208157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.208750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.209044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.210146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:40.223021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:40.224261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:40.225303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:40.227649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 516/1036] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:40.234719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.324618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4680f2-7aa0-4ec7-b42a-c7f3fb164291 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.326171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.350926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ea2b-ec31-456e-b0ea-78f336c3ba71 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.363739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.364369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.364648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.364917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.377107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.378540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.388636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.389433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.389607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.389884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.390732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:40.391121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.393184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:40.394848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:40.395829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:40.396714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:40.396714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:40.396714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:40.396714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 521/1037] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:40.400721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 517/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:40.401756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.406348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.418735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.511368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91019fa1-079d-4c7b-911d-6cac2c070341 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.520013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acec7-d50d-4295-a88d-349231dc27a8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.548096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.548703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.548984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.549259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.549902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:40.550203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.551621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.559141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.559775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.560054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.560323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.561067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:40.561367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.562728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.565355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 522/1039] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:40.569455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 518/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:40.570649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.575011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.586576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.666177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.678861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5778474-203d-488f-bd5c-2a24d2dd1d42 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.692951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ca2a5-5110-480f-8cf2-edc35b9d9eb2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.716866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.717511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.717792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.718071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.727580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.729035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.738255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.740282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.740561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.740830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.741543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] RBAC: Adding query filter params (interface=public, service_id=8012bc68da5d4779beb2cf08fdd1e95d, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:40.741849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.743240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:40.743337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:40.744147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:40.750164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 523/1041] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:40.750675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 519/1042] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/endpoints?service_id=8012bc68da5d4779beb2cf08fdd1e95d&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:40.754827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.756070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.843717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:40.858508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f3608-8a32-43ca-811a-0172832ae2e6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.875468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32456d-aefa-49c5-bfcb-f9998fca0120 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:40.896169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.896780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.897182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.897398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.897911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.899463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.906206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 520/1043] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:05:40.923343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:40.923980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:40.924272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:40.924528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:40.925236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:40.925538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:40.927012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c672c67-959b-4447-824b-8e52a6a715ea None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:40.945793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 524/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:40.947737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:40.950722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:41.078219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c01bf1-6bae-420a-8c47-b26991d86045 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:41.116252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:41.116869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:41.117152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:41.117484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:41.118248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] RBAC: Adding query filter params (role.id=9d8dcbdd28914d56b935a45e3063c79b, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:41.118565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:41.120036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782a7ce-3991-4a0e-935f-903c2a994def None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:41.126008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 521/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:40 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=9d8dcbdd28914d56b935a45e3063c79b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:42.052788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657889-7db2-44b1-b918-a5e6fdc0df61 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.053313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657889-7db2-44b1-b918-a5e6fdc0df61 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.053393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657889-7db2-44b1-b918-a5e6fdc0df61 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.054832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 525/1046] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:42.059884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.060306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.060432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.078985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:42.080192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.150077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:42.159283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 522/1047] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:42.428534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e657889-7db2-44b1-b918-a5e6fdc0df61 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.534494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e657889-7db2-44b1-b918-a5e6fdc0df61 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.568509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.569130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.569408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.569721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.571596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:42.582294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:42.587006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:42.587964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:42.588824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:42.591581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 526/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:42.594324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.715936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6605e-82d6-4c2f-9e3b-5ee5d1553c17 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.753604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.754208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.754495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.754751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.755442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:42.755748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.757117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2715d-32a6-4674-a586-0a25b28f204d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.766712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 523/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:42.773011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.848912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaf6c-a952-4770-8bb3-7dd1b23057c3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.849244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaf6c-a952-4770-8bb3-7dd1b23057c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.849484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aaf6c-a952-4770-8bb3-7dd1b23057c3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.850971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 524/1050] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:42.856157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683de3-6e3a-44ff-9d7b-17f3856bff19 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.856157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683de3-6e3a-44ff-9d7b-17f3856bff19 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.856684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683de3-6e3a-44ff-9d7b-17f3856bff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.875982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bc683de3-6e3a-44ff-9d7b-17f3856bff19 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:42.877183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bc683de3-6e3a-44ff-9d7b-17f3856bff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:42.882172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2d4a8-8ac2-4c63-8f1f-4e34adc73609 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:42.923198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:42.923847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:42.924124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:42.924394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] PATH_INFO: `/v3/users/b4ae56186cbe42188985ca29a302d12c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:42.939284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:42.941532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] RBAC: Authorizing `identity:get_user(user_id=b4ae56186cbe42188985ca29a302d12c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:42.942993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:42.945034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 527/1051] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/users/b4ae56186cbe42188985ca29a302d12c => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:42.950591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:42.959575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 525/1052] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:43.007271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.057321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237444be-00cb-4a30-b46d-c11989094601 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.092608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.093222 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.093506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.093784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.097313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0bdf-ef49-45ea-933c-f8de44f131f0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.097790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0bdf-ef49-45ea-933c-f8de44f131f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.097871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0bdf-ef49-45ea-933c-f8de44f131f0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.099359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 526/1053] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:05:43.104312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.104648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.104857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.124875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:43.126719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:43.167599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.169262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.197658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 527/1054] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:43.225357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.228922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.283359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 528/1055] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:05:43.284340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.337024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977954a-3c85-48aa-ad8b-533e1e367b87 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.356783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.375555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.376216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.376500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.376776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.377364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.378766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:43.387437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:43.388456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:43.389302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 528/1056] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:43.394326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.399820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1f60-4e83-47ed-9518-30e8ecaaaee5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.436399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.437034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.437311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.437582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.438714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.440179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.441960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 529/1057] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:43.447702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.470232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.504368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44070573-4aa6-444e-8d4f-0ca8a6133455 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.538079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.538595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.538873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.539142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.539792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:43.540091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.541457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.546268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 529/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:43.547037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.555468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4ecf7-c125-497e-91bb-d91c6550cc8d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.589445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.590179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.590567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.590946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.591834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=4007b192d8c8436395b7e7827d951cf1, user.id=b4ae56186cbe42188985ca29a302d12c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:43.592236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.592733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.593918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.600515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 530/1059] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/role_assignments?user.id=b4ae56186cbe42188985ca29a302d12c&scope.project.id=4007b192d8c8436395b7e7827d951cf1&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:43.604229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.667483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf1d9-e78b-482f-a08c-5e23cabb78dc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.705886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.706540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.706827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.707078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.712949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.720323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:43.725651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:43.726867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:43.727812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 530/1060] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:43.732992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.739190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8d509-8580-4eaf-aca1-c2a7a65de93f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.775708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.776319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.776598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.776879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.777511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] RBAC: Adding query filter params (type=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:43.777821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.779185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.784240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 531/1061] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:43.789949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.852474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283354-4bf9-486f-af69-28b20de69015 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.860825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:43.890999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.891707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.891983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.892253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.892920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:43.893223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.894597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.901644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe065b1-d946-401c-9311-0dbb8cbf1652 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:43.903258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 531/1062] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:43.908946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:43.939953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:43.940775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:43.941136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:43.941574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:43.942321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:43.943893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:43.975253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 532/1063] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:05:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:05:44.030330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a594a-c3ab-4b9d-951b-e1a5a474e2d1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.076504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.077062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.077346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.077624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.089795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.091316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:44.107616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:44.109203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:44.110796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:44.110796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:44.110796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:44.110796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 532/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:44.115742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.120633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:44.231779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280875f9-5bf4-4a66-9474-1c550b1e17fa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.268317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.268923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.269205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.269478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.270120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:44.270423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.271824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.284544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 533/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:44.289929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.309189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:44.394098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d571b-2356-4f6c-89bc-d2f19334381f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.427629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.428425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.428802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.429176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.437009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.438634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:44.450067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:44.451202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:44.452368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:44.454474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 533/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:44.460176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.525845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:44.567465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8515f-8489-4091-bdef-aa6fb955fb13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.602035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.602726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.603080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.603449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.604245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:44.604632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.606111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.630206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 534/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:44.631491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.716484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:44.742400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffeba-cd36-4ce4-8f40-6cf6d76be41d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.782353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.782975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.783293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.783572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.794327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.795798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:44.810206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:44.811159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:44.812360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:44.812360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:44.812360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:44.816741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 534/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:44.817683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:44.927504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6926a-7bb2-48dd-a84f-d43201f1e8eb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:44.950839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:44.963998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:44.964777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:44.965022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:44.965363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:44.966064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:44.966435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:44.967886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:44.984746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 535/1069] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:44.986470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.093341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27114-bc67-45fe-a5c3-dd1bd409a027 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.121970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.133338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.133967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.134245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.134518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.141618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.143055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:45.154575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:45.155726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:45.156625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 535/1070] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:45.163956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.268655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93b51-23f7-4510-bf3b-995693ad6b0c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.302826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.303473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.303757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.304036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.304723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:45.305032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.306454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644ea14-08b5-4df4-9bf9-6421a793b56f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.318669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 536/1071] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:45.324449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.428152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30522a96-2e6a-4874-bd9f-4c23ea03ed29 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:45.462392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:45.463057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.463367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.463642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/9d8dcbdd28914d56b935a45e3063c79b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.465789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=5ce87b2207504e839bcd544faa3aa7ad, role_id=9d8dcbdd28914d56b935a45e3063c79b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:45.467246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc491782-0df3-4ce9-8f7d-5656f7bc0b4c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:45.482808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 536/1072] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:45 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/5ce87b2207504e839bcd544faa3aa7ad/roles/9d8dcbdd28914d56b935a45e3063c79b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:05:45.512877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240ea0-dcd0-42f7-8527-27ac21bc3fea None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.513228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240ea0-dcd0-42f7-8527-27ac21bc3fea None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.513443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240ea0-dcd0-42f7-8527-27ac21bc3fea None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.514999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 537/1073] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:45.522785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4d76-36b8-4d08-8493-f04669ff1d57 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.523073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4d76-36b8-4d08-8493-f04669ff1d57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.523304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4d76-36b8-4d08-8493-f04669ff1d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.544795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47aa4d76-36b8-4d08-8493-f04669ff1d57 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:45.546791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47aa4d76-36b8-4d08-8493-f04669ff1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:45.608209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.642160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 537/1074] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:45.911852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240ea0-dcd0-42f7-8527-27ac21bc3fea None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:45.964042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:45.980505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c22-2411-47f5-987e-4301ff07c1e5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.980821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c22-2411-47f5-987e-4301ff07c1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.981060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c22-2411-47f5-987e-4301ff07c1e5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:45.982410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 538/1075] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:45.987886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:45.988173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:45.988451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.006988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:46.008216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:46.028639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240ea0-dcd0-42f7-8527-27ac21bc3fea None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.064036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.064570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.064852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.065126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.076271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:46.079683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 539/1076] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:46.081543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:46.082516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:46.083451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 538/1077] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:46.088684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.190672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1cd80-ffa4-4835-87ce-2b085a66e3d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.200106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.230057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.230665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.230943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.231211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.231897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:46.232198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.233564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.247297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 540/1078] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:46.248572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.352019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d0110-d75c-4759-8653-09c1c9aa36c5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.353564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.355028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.391886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868ec28-325e-48b9-835f-54787599085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.392517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868ec28-325e-48b9-835f-54787599085b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.392796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868ec28-325e-48b9-835f-54787599085b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.393064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868ec28-325e-48b9-835f-54787599085b None admin] PATH_INFO: `/v3/users/b4ae56186cbe42188985ca29a302d12c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.394939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868ec28-325e-48b9-835f-54787599085b None admin] RBAC: Authorizing `identity:get_user(user_id=b4ae56186cbe42188985ca29a302d12c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.396382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868ec28-325e-48b9-835f-54787599085b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.398106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 539/1079] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/users/b4ae56186cbe42188985ca29a302d12c => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:46.403696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868ec28-325e-48b9-835f-54787599085b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.467345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04e0f2-6d24-4ce9-8790-53e0588e2848 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.505443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.506045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.506435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.506709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.510068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868ec28-325e-48b9-835f-54787599085b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.548971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.549606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.549886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.550158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.551311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.551645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.552738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.554777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 540/1080] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:46.560289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.589023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.590655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.608226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.673308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2dd5c-1e7a-4fa6-9f7b-f88f7954f204 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.712028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.712218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 541/1081] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:05:46.712489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.712771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.713114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1/users/b4ae56186cbe42188985ca29a302d12c/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.715039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] RBAC: Authorizing `identity:create_grant(project_id=4007b192d8c8436395b7e7827d951cf1, user_id=b4ae56186cbe42188985ca29a302d12c, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.716486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.717894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:46.731248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 541/1082] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:46 2022] PUT /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1/users/b4ae56186cbe42188985ca29a302d12c/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:05:46.797439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:46.834441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c50f1-4bf6-494a-b97d-f25ee4268c75 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:46.877609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:46.878250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:46.878526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:46.878799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:46.890020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:46.891577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-69030431-01df-4099-9a07-48958c7a6686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:46.906879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:46.907974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:46.908866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:46.908866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:46.908866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:46.908866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 542/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:46.914261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:47.019403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:47.023456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac771f8-bc5d-4c5d-96f2-82e4fa1b3427 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:47.062515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:47.063194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:47.063506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:47.063782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:47.064435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:47.064741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:47.066105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a016ef-220d-499c-99b4-ecde4da6ad79 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:47.078224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 542/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:47.447863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507e652-86a2-4892-9fe2-bfbe668dcffc None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:47.448385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507e652-86a2-4892-9fe2-bfbe668dcffc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:47.448509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507e652-86a2-4892-9fe2-bfbe668dcffc None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:47.449749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 543/1085] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:47.455530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:47.455953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:47.456088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:47.474381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:47.475489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:47.541986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 543/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:47.810143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507e652-86a2-4892-9fe2-bfbe668dcffc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:47.920306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507e652-86a2-4892-9fe2-bfbe668dcffc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:47.943022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:47.955036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:47.955682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:47.955966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:47.956240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:47.966764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:47.972077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:47.973053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:47.974546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 544/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 30 16:05:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:47.979145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.070000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:48.082257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b812b-92c1-45c6-93dc-6844a50f7f5a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.117280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.117894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.118170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.118441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.119097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:48.119429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:48.120785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:48.129184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 544/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:05:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:48.134885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.234014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a454b2f-f710-451f-91c2-e960dfd3e9c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.270641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.271283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.271578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.271855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.282472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:48.283950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:48.299038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:48.300291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:48.301879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:48.301879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:48.301879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:48.301879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 545/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:48.306077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.406566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4a92e-d30f-4d64-874c-887e04566acb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.439767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.440363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.440639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.440908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.441535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:48.441833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:48.443171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:48.455120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 545/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:48.460713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.502263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:48.570002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c119ad-bf4a-422f-ba96-c059a79df436 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.603802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.604412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.604696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.604967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.612851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:48.614301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5523705f-cf1f-4225-a533-d0f965dee980 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:05:48.625453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:48.626958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:48.627957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 546/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 30 16:05:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:05:48.632876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.648186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:48.668281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea07f3-f5aa-4dba-a596-bd8ca8c67153 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.668566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea07f3-f5aa-4dba-a596-bd8ca8c67153 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.668839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea07f3-f5aa-4dba-a596-bd8ca8c67153 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.670166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 547/1092] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:48.675044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.675345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.675614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.693989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:48.695256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:48.743863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c800a-3ee9-45ad-91ef-3f7141af7c5c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.767691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 548/1093] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:05:48.783823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.784435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.784714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.784989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:48.785755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:48.786058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:48.787510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:48.806069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 546/1094] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 30 16:05:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:48.811906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:48.838065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:48.913309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12044f-5639-4d2c-a5d0-105ee936757b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:48.948251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:48.948834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:48.949053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:48.949329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.015654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.017130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.040386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.057533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.123762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 549/1095] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:05:49.130523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.132255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.166611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb07a0e-c433-4cf6-9221-c4320d6c15d6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.202190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.202732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.203011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.203310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.213602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:05:49.218326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:49.219363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:05:49.220223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 547/1096] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:49.225580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.233138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3016d4e-7309-49cf-9e69-3728307e349c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.257457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.268144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.268763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.269057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.269326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.279157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.280614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:05:49.296019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:49.296976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:05:49.298579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:05:49.298579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:05:49.298579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:49.298579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 550/1097] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:49.303281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.327954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2b9ca-addc-443e-a79f-ace5538a769a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.354248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.363158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.363788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.364079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.364352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.365015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:49.365321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.366685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.375841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 548/1098] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:49.381958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.403892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18fa8d-1add-49e7-aaaf-235ed2f9a3ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.456240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.456759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.457042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.457319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.457969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:49.458400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.459701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.477148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 551/1099] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:49.478295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.487840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb910498-4234-4943-9b62-69f25d5028e2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.514722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.522607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.523269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.523559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.523840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] PATH_INFO: `/v3/users/b4ae56186cbe42188985ca29a302d12c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.525004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] RBAC: Authorizing `identity:get_user(user_id=b4ae56186cbe42188985ca29a302d12c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.526449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.528079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 549/1100] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/users/b4ae56186cbe42188985ca29a302d12c => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:49.534393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.579889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e9a9e-f75d-4fd7-a7d2-18859b37c679 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.593419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.616175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.616803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.617080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.617353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.624154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.625609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.636871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19290a27-7cc1-435b-a18e-7cee17391a56 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:05:49.637712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:05:49.638630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:05:49.639595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 552/1101] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:05:49.645072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.674274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.674873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.675150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.675446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.741113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.747819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a183f-8dff-4904-aa07-192c2d2cdf25 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.748965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.750436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.782786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.783512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.783707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.783986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.784679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:49.784986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.786474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.798597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 553/1102] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:49.804908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.807799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:49.851464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 550/1103] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:05:49.857398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:49.907611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f4574-eb46-4dac-91c2-fade34a3f52c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:49.941744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:49.942337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:49.942614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:49.942879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:49.943628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] RBAC: Adding query filter params (role.id=9d8dcbdd28914d56b935a45e3063c79b, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=5ce87b2207504e839bcd544faa3aa7ad) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:49.943934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:49.945354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:49.950971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 554/1104] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/role_assignments?user.id=5ce87b2207504e839bcd544faa3aa7ad&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=9d8dcbdd28914d56b935a45e3063c79b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:49.962743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09d3fb-c14f-456b-b3af-e2668b8ee6b8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.001343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.001907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.002192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.002472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.003748 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.005226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932e61-26df-487b-8778-a91c7bf7cc4b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.006959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 551/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:05:49 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:05:50.013420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:50.038958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:50.121139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56593924-bdd4-470b-9b32-803a9209634f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:50.154775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:50.155403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:50.155680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:50.155946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:50.157222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=4007b192d8c8436395b7e7827d951cf1, user.id=b4ae56186cbe42188985ca29a302d12c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:50.157537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:50.158965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be31fe-a4cb-47f5-b90d-90458b347fae None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:50.165189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 555/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:50 2022] GET /identity/v3/role_assignments?user.id=b4ae56186cbe42188985ca29a302d12c&scope.project.id=4007b192d8c8436395b7e7827d951cf1&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:52.114073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa43b1-9fd2-47af-b477-f6c90193611f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:52.114605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa43b1-9fd2-47af-b477-f6c90193611f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:52.114605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa43b1-9fd2-47af-b477-f6c90193611f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:52.115978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 552/1107] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:52.121131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:52.121519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:52.121695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:52.140172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:52.141378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:52.206404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 556/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:52.474602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa43b1-9fd2-47af-b477-f6c90193611f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:52.502578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:52.575751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa43b1-9fd2-47af-b477-f6c90193611f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:52.608701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:52.609317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:52.609603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:52.610435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:52.611837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:52.613264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92bce-5c48-41f6-bc6c-cc2fd52a0dac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:52.614812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 553/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:05:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:52.620574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:52.635652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:52.720510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7e41-bfd2-4128-bc5a-47c221a44dff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:52.754052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:52.754582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:52.754864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:52.755134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:52.755821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:52.757256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a3321-b1c2-48c0-b163-77e065b5d939 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:52.778612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 557/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:05:52 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:05:54.741445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d3e1a-fbfe-49d4-af5b-36694fa3d1c4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:54.742017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d3e1a-fbfe-49d4-af5b-36694fa3d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:54.742017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d3e1a-fbfe-49d4-af5b-36694fa3d1c4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:54.743367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 554/1111] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:54.748558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:54.748964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:54.749116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:54.767539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:54.768701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:54.831640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:54.839253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 558/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:55.110021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d3e1a-fbfe-49d4-af5b-36694fa3d1c4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:55.205324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:55.212738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d3e1a-fbfe-49d4-af5b-36694fa3d1c4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:55.246820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:55.247458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:55.247739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:55.248011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] PATH_INFO: `/v3/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:55.249386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] RBAC: Authorizing `identity:get_role(role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:55.250770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:55.252114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 555/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:55 2022] GET /identity/v3/roles/1927f695a15842babcb8a7b304466dbb => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:55.257919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:55.357822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0f645-636e-4bbd-a825-a8f989b6d24f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:55.391670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:55.392227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:55.392510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:55.392788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] PATH_INFO: `/v3/users/fd11d6a0466b441e8b66b8088340dd1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:55.405522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] RBAC: Authorizing `identity:get_user(user_id=fd11d6a0466b441e8b66b8088340dd1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:55.407003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:55.408636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 559/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:55 2022] GET /identity/v3/users/fd11d6a0466b441e8b66b8088340dd1a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:55.414416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:55.511534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913f21-8cf5-456a-8b72-c7d44930a71f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:55.545379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:55.545973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:55.546254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:55.546527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:55.575888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:55.613730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:55.615164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:55.710239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 556/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:05:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:05:55.716405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:55.774179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:55.814692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b652e0-cce5-40db-8a27-b8ef1949c770 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:55.848221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:55.848838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:55.849121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:55.849395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:55.850500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:55.851961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a76b39-df8b-4f62-a18a-061c8c9d781f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:55.853593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 560/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:05:55 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:55.859878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:55.920117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:55.958181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ea1ba-2982-4acf-8f5b-6125891db27c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:55.991643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:55.992234 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:55.992510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:55.992780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:55.993485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=4007b192d8c8436395b7e7827d951cf1, user.id=fd11d6a0466b441e8b66b8088340dd1a) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:05:55.993791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:55.995197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3d42c-7cef-4f46-8415-5d0f79aedb67 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:56.000427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 557/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:05:55 2022] GET /identity/v3/role_assignments?user.id=fd11d6a0466b441e8b66b8088340dd1a&scope.project.id=4007b192d8c8436395b7e7827d951cf1&role.id=1927f695a15842babcb8a7b304466dbb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:57.879775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ee0fa-4009-4bcd-9091-a274ce054e7c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:57.880344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ee0fa-4009-4bcd-9091-a274ce054e7c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:57.880344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ee0fa-4009-4bcd-9091-a274ce054e7c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:57.881671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 561/1118] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:05:57.887035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:57.887474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:57.887624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:57.906129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:05:57.907307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:05:57.971747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 558/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:05:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:05:58.243741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ee0fa-4009-4bcd-9091-a274ce054e7c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:58.355359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ee0fa-4009-4bcd-9091-a274ce054e7c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:58.384098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:58.394135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:58.394764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:58.395045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:58.395350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] PATH_INFO: `/v3/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:58.396853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] RBAC: Authorizing `identity:get_role(role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:58.398232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:58.399761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 562/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:58 2022] GET /identity/v3/roles/1927f695a15842babcb8a7b304466dbb => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:58.405448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:58.461988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:58.508710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8ded7-1bc1-4b0e-a041-0d3b30d82c9b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:58.543269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:58.543899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:58.544176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:58.544446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] PATH_INFO: `/v3/users/fd11d6a0466b441e8b66b8088340dd1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:58.545539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] RBAC: Authorizing `identity:get_user(user_id=fd11d6a0466b441e8b66b8088340dd1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:58.546962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:58.548600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 559/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:05:58 2022] GET /identity/v3/users/fd11d6a0466b441e8b66b8088340dd1a => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:58.554494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:58.652670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a34cbc-21ac-4b4c-84ed-375d220b52ff None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:58.685741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:58.686359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:58.686641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:58.686917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:58.688621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:58.690057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea130b-f0b0-4028-94b5-085481e8603f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:58.691715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 563/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:05:58 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:05:58.697864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:05:58.797678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4b76-92b2-4ff8-93c7-36829d4f3c25 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:05:58.821147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:05:58.831781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:05:58.832428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:05:58.832706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:05:58.832977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1/users/fd11d6a0466b441e8b66b8088340dd1a/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:05:58.835062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] RBAC: Authorizing `identity:create_grant(project_id=4007b192d8c8436395b7e7827d951cf1, user_id=fd11d6a0466b441e8b66b8088340dd1a, role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:05:58.836500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058cbc6-cc7c-4c72-9ea8-8f90272d1501 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:05:58.848926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 560/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:05:58 2022] PUT /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1/users/fd11d6a0466b441e8b66b8088340dd1a/roles/1927f695a15842babcb8a7b304466dbb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:00.738888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97369-84cb-4cf4-bced-de492a4053d4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.739557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97369-84cb-4cf4-bced-de492a4053d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.739557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97369-84cb-4cf4-bced-de492a4053d4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.740981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 564/1124] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:00.746437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:00.746883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:00.746988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:00.765797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:00.767078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:00.835396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 561/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:01.108793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97369-84cb-4cf4-bced-de492a4053d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.171858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.212613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97369-84cb-4cf4-bced-de492a4053d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.250128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.250783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.251069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.251876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] PATH_INFO: `/v3/roles/1927f695a15842babcb8a7b304466dbb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.253344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] RBAC: Authorizing `identity:get_role(role_id=1927f695a15842babcb8a7b304466dbb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.254862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.256253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 565/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/roles/1927f695a15842babcb8a7b304466dbb => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:01.262969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.361469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ec32c-efd2-4f56-b4d2-9e251ffe7241 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.375069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.397035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.397654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.397937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.398211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] PATH_INFO: `/v3/users/fd11d6a0466b441e8b66b8088340dd1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.399355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] RBAC: Authorizing `identity:get_user(user_id=fd11d6a0466b441e8b66b8088340dd1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.400788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.402368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 562/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/users/fd11d6a0466b441e8b66b8088340dd1a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:01.409016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.520552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593239b1-a224-4419-9db5-9e0b9e993c3a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.550455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.557824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.558448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.558727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.558997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.628475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.629670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.725450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 566/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:01.731501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.828085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857dd07-3baa-42ca-9695-7c912aefb237 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:01.862023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:01.862646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:01.862930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:01.863210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] PATH_INFO: `/v3/projects/4007b192d8c8436395b7e7827d951cf1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:01.864361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] RBAC: Authorizing `identity:get_project(project_id=4007b192d8c8436395b7e7827d951cf1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:01.865782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3fb26-f4eb-4818-8e07-2684bb7b8b45 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:01.867430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 563/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/projects/4007b192d8c8436395b7e7827d951cf1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:01.873667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:01.905270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:01.974034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace79c7-c25a-4e24-9923-3e77392bd8bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:02.007989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:02.008582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:02.008860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:02.009131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:02.009848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] RBAC: Adding query filter params (role.id=1927f695a15842babcb8a7b304466dbb, scope.project.id=4007b192d8c8436395b7e7827d951cf1, user.id=fd11d6a0466b441e8b66b8088340dd1a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:02.010158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:02.011618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dc3e7-cbc5-42d6-87be-97c164fa7be8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:02.017301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 567/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:06:01 2022] GET /identity/v3/role_assignments?user.id=fd11d6a0466b441e8b66b8088340dd1a&scope.project.id=4007b192d8c8436395b7e7827d951cf1&role.id=1927f695a15842babcb8a7b304466dbb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:04.032920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfbe8-f1bc-4a47-9531-242ebe85730f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.033477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfbe8-f1bc-4a47-9531-242ebe85730f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.033477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfbe8-f1bc-4a47-9531-242ebe85730f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.034854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 564/1131] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:04.040544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.040979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.041075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.058858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:04.060086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:04.125939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 568/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:04.418145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfbe8-f1bc-4a47-9531-242ebe85730f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.469051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:04.521740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfbe8-f1bc-4a47-9531-242ebe85730f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.556070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.556694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.556980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.557258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.558375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.559833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade25c-1249-4f20-b1cd-a991073b859c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.561394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 565/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:04.567422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:04.591437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:04.667479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc0859-22bb-42c4-9650-bb25c7a3aa81 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:04.700444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:04.700976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:04.701259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:04.701536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:04.702182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:04.703659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef499b7-7160-4b64-b467-d0be6700fd63 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:04.730971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 569/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:06:04 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:06.681879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6b7f-b777-4335-bb35-78a8c21ebb97 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:06.682342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6b7f-b777-4335-bb35-78a8c21ebb97 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:06.682428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6b7f-b777-4335-bb35-78a8c21ebb97 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:06.683800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 566/1135] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:06.689021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:06.689490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:06.689578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:06.707561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:06.708748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:06.774949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 570/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:07.051812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f6b7f-b777-4335-bb35-78a8c21ebb97 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:07.152096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f6b7f-b777-4335-bb35-78a8c21ebb97 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:07.169502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:07.186674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:07.187319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:07.187605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:07.187880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:07.189025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:07.190443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d39ca-89d5-4787-ac9c-8148ad1eb307 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:07.192036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 567/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:06:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:07.197794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:07.206375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:07.295934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4f9a4-a5ca-408d-85a9-bf07b824f4de None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:07.330133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:07.330660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:07.330939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:07.331209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:07.331884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:07.333317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8e843-8c3b-46a1-845a-c93facf41101 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:07.354467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 571/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:06:07 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:09.378205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9970c7-de7c-4a8f-994f-b0705da6b0c5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:09.378767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9970c7-de7c-4a8f-994f-b0705da6b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:09.378767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9970c7-de7c-4a8f-994f-b0705da6b0c5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:09.380112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 568/1139] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:09.385306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:09.385704 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:09.385862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:09.404313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:09.405498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:09.460566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:09.471835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 572/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:09.737577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9970c7-de7c-4a8f-994f-b0705da6b0c5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:09.839553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9970c7-de7c-4a8f-994f-b0705da6b0c5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:09.873353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:09.873971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:09.874253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:09.874527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:09.885362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:09.890146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:09.891513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:09.892735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 569/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:09.897664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:09.994188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201622e-0a2f-4d51-80fe-b53c0037cf79 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:10.026868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:10.027484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:10.027765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:10.028036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:10.028699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:10.028995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:10.030367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:10.038410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 573/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:10.044350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:10.093043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:10.147521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530cbf5-fe2f-415a-8206-0d8c9855cc7e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:10.181738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:10.182362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:10.182642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:10.182916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] PATH_INFO: `/v3/users/a6fd12b17fd94fd695011567a6ce3a7c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:10.196473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fd12b17fd94fd695011567a6ce3a7c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:10.197916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:10.199578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 570/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:10 2022] GET /identity/v3/users/a6fd12b17fd94fd695011567a6ce3a7c => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:10.206084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:10.265000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:10.310871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4343-53d0-43f6-8e17-6e8b1df1693b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:10.345684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:10.346289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:10.346572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:10.346844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:10.417835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:10.419317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:10.473650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:10.519683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 574/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:10.525665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:10.628554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963838-5c5c-4c71-85fd-52d7b2c5a749 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:10.630995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:10.666601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:10.667135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:10.667450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:10.667720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] PATH_INFO: `/v3/projects/b9c2444fdc724472a50818afa012ef26` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:10.668856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c2444fdc724472a50818afa012ef26)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:10.670291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41106837-1c69-46e5-b11c-4d87b0cde3b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:10.671977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 571/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:10 2022] GET /identity/v3/projects/b9c2444fdc724472a50818afa012ef26 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:10.678680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:10.778338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b1dd4-b677-4210-8f6f-46bb9e17f6cd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:10.812314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:10.812916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:10.813196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:10.813468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:10.814738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b9c2444fdc724472a50818afa012ef26, user.id=a6fd12b17fd94fd695011567a6ce3a7c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:10.815049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:10.816514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2b44-30c7-4969-b483-67fcd59e6fb1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:10.821952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 575/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:06:10 2022] GET /identity/v3/role_assignments?user.id=a6fd12b17fd94fd695011567a6ce3a7c&scope.project.id=b9c2444fdc724472a50818afa012ef26&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:12.760142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f42e9-a6ee-481d-8c75-c0390cd06891 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:12.760667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f42e9-a6ee-481d-8c75-c0390cd06891 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:12.760667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f42e9-a6ee-481d-8c75-c0390cd06891 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:12.762035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 572/1147] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:12.767697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:12.768130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:12.768255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:12.786304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:12.787531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:12.801040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:12.854294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 576/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:13.120510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f42e9-a6ee-481d-8c75-c0390cd06891 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:13.221289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f42e9-a6ee-481d-8c75-c0390cd06891 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:13.255004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:13.255647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:13.255928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:13.256203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:13.266098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:13.270826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:13.271934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:13.272777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 573/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:13.278030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:13.375165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10cf02-7191-4ee9-865a-f72094727d98 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:13.408546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:13.409159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:13.409437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:13.409706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:13.410362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:13.410661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:13.412058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:13.420279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 577/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:13.426704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:13.457301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:13.525267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567cb61-133f-45d4-afa7-60f898281929 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:13.559477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:13.560092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:13.560375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:13.560653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] PATH_INFO: `/v3/users/a6fd12b17fd94fd695011567a6ce3a7c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:13.561766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] RBAC: Authorizing `identity:get_user(user_id=a6fd12b17fd94fd695011567a6ce3a7c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:13.563247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:13.564836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 574/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:13 2022] GET /identity/v3/users/a6fd12b17fd94fd695011567a6ce3a7c => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:13.570646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:13.583448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:13.669636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990a4eb-6e9b-4d96-bad3-cf5d4e544ddf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:13.701718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:13.702345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:13.702637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:13.702913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] PATH_INFO: `/v3/projects/b9c2444fdc724472a50818afa012ef26` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:13.704587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] RBAC: Authorizing `identity:get_project(project_id=b9c2444fdc724472a50818afa012ef26)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:13.706026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ef19-88a5-48a9-b034-443caee5de5c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:13.707689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 578/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:13 2022] GET /identity/v3/projects/b9c2444fdc724472a50818afa012ef26 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:06:13.713744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:13.807641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:13.810577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae5f-a128-4acd-a12a-bc03da9225dc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:13.845290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:13.845968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:13.846253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:13.846532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] PATH_INFO: `/v3/projects/b9c2444fdc724472a50818afa012ef26/users/a6fd12b17fd94fd695011567a6ce3a7c/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:13.848403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9c2444fdc724472a50818afa012ef26, user_id=a6fd12b17fd94fd695011567a6ce3a7c, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:13.849824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d42351-4767-4c68-ae6f-3ecbe0df60c4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:13.862997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 575/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:06:13 2022] PUT /identity/v3/projects/b9c2444fdc724472a50818afa012ef26/users/a6fd12b17fd94fd695011567a6ce3a7c/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:15.718125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29095735-868b-4b7a-aaa8-ffee78dacfc8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:15.718600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29095735-868b-4b7a-aaa8-ffee78dacfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:15.718705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29095735-868b-4b7a-aaa8-ffee78dacfc8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:15.720600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 579/1154] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:15.725851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:15.726224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:15.726410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:15.745296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:15.746509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:15.811875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 576/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:16.077850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29095735-868b-4b7a-aaa8-ffee78dacfc8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.156238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.178685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29095735-868b-4b7a-aaa8-ffee78dacfc8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.211372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.211984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.212266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.212539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.222814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:16.227566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:16.228643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:06:16.229685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 580/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:16.234841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.323555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.333585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66e19-d6d7-4ea1-8e58-40b7c18ab13d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.369057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.369655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.369932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.370201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.370850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:16.371147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.372525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.380597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 577/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:16.386756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.492253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eba4e6-f65e-4143-b087-c9a616f7fb79 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.524646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.525268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.525550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.525829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] PATH_INFO: `/v3/users/a6fd12b17fd94fd695011567a6ce3a7c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.526945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fd12b17fd94fd695011567a6ce3a7c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.528412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.529986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 581/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/users/a6fd12b17fd94fd695011567a6ce3a7c => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:16.536479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.635653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb8a3-9c57-4db6-8eab-060676fecc1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.669505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.670107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.670391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.670663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.691367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.738994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.740556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.836249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 578/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:16.842203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:16.852177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:16.944040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedc327-2c39-4848-aeb8-e87b7f12c534 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:16.977139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:16.977757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:16.978042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:16.978318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] PATH_INFO: `/v3/projects/b9c2444fdc724472a50818afa012ef26` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:16.979449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c2444fdc724472a50818afa012ef26)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:16.980888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389e1d4-a278-4c3b-9cc7-c6347e732152 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:16.982505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 582/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/projects/b9c2444fdc724472a50818afa012ef26 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:16.988964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:17.044219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:17.091210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3ab8-fbcf-49d8-b8cd-0d60a98ad895 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:17.124133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:17.124754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:17.125035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:17.125307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:17.126539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b9c2444fdc724472a50818afa012ef26, user.id=a6fd12b17fd94fd695011567a6ce3a7c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:17.126846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:17.128287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ad8a-ee8e-498d-a3ce-0e951d26ccc8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:17.133882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 579/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:06:16 2022] GET /identity/v3/role_assignments?user.id=a6fd12b17fd94fd695011567a6ce3a7c&scope.project.id=b9c2444fdc724472a50818afa012ef26&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:19.059753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38d06e-90f7-4861-a736-04d60e1bcac0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.060323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38d06e-90f7-4861-a736-04d60e1bcac0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.060323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38d06e-90f7-4861-a736-04d60e1bcac0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.061653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 583/1162] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:19.067175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.067600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.067768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.086110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:19.087326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:19.152647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 580/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:19.429453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38d06e-90f7-4861-a736-04d60e1bcac0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.506236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:19.532667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38d06e-90f7-4861-a736-04d60e1bcac0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.565558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.566152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.566437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.566713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.638712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.640208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.714368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:19.739735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 584/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:19.745786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:19.790629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:19.840875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622ac31-13b9-4f93-a952-bd9f46c54f75 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:19.873771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:19.874381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:19.874656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:19.874924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:19.881501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:19.882932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-aed2f6ca-3247-4737-a644-edfd2744fa7e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:19.895592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:19.896822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:19.897901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 581/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:19.902851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:20.006071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049c1ab-2cf7-4f18-ae60-100aabecdd0b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:20.038892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:20.039510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:20.039790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:20.040059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:20.040689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:20.040993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:20.042351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce201-cadb-47c3-a186-4fa243147f43 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:20.049351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 585/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 30 16:06:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:21.931104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ced-b307-44b4-a59c-79a51bdd5895 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.931677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ced-b307-44b4-a59c-79a51bdd5895 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.931677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ced-b307-44b4-a59c-79a51bdd5895 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.933083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 582/1167] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:21.938873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8874-e5c2-4d40-b025-bdf05875176b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:21.939567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8874-e5c2-4d40-b025-bdf05875176b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:21.939567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8874-e5c2-4d40-b025-bdf05875176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:21.961910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0a5a8874-e5c2-4d40-b025-bdf05875176b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:21.963098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0a5a8874-e5c2-4d40-b025-bdf05875176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:21.993866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:22.036789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 586/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:22.303256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5ced-b307-44b4-a59c-79a51bdd5895 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:22.403619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5ced-b307-44b4-a59c-79a51bdd5895 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:22.410604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:22.437377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:22.437972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:22.438769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:22.439051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:22.439761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:22.441122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93cb62-7bd9-40db-801f-cc6b1a7e3438 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:22.457295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 583/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Tue Aug 30 16:06:22 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:24.412235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc36d6-37d2-4bad-9af7-782c72a78647 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.412873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc36d6-37d2-4bad-9af7-782c72a78647 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.412873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc36d6-37d2-4bad-9af7-782c72a78647 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.414132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 587/1170] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:24.419210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.419627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.419804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.437760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:24.438980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:24.509359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 584/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:24.783390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc36d6-37d2-4bad-9af7-782c72a78647 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:24.885710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc36d6-37d2-4bad-9af7-782c72a78647 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:24.920111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:24.920726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:24.921006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:24.921281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] PATH_INFO: `/v3/domains/02ece6a9db264c6a8f249d65a9c08391` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:24.931510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=02ece6a9db264c6a8f249d65a9c08391)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:24.932949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2874-5b33-478f-9943-1340c65c0048 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:24.934525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 588/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Aug 30 16:06:24 2022] GET /identity/v3/domains/02ece6a9db264c6a8f249d65a9c08391 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:24.940435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:25.031816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:25.038997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4861fc0-5ab8-4bbb-8eb5-30c59773c3c3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:25.071373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:25.071961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:25.072239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:25.072511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:25.073150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:25.074560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4936e-d9c8-42a7-901e-4ae9e32e607f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:25.100959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 585/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:06:24 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:27.042903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556ef86-8239-4389-8589-6bd2e80ad38a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:27.043486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556ef86-8239-4389-8589-6bd2e80ad38a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:27.043486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556ef86-8239-4389-8589-6bd2e80ad38a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:27.044814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 589/1174] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:27.050192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:27.050635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:27.050741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:27.070042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:27.071253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:27.137759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 586/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:27.414447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556ef86-8239-4389-8589-6bd2e80ad38a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:27.443992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:27.517391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556ef86-8239-4389-8589-6bd2e80ad38a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:27.550342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:27.550959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:27.551264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:27.551542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] PATH_INFO: `/v3/domains/02ece6a9db264c6a8f249d65a9c08391` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:27.552662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=02ece6a9db264c6a8f249d65a9c08391)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:27.554107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8b53-b626-4342-b255-b7e8cfc664e8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:27.555701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 590/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Aug 30 16:06:27 2022] GET /identity/v3/domains/02ece6a9db264c6a8f249d65a9c08391 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:27.561404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:27.639569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:27.663425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c317f34-e839-4db4-aff2-da6fb3fc74f6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:27.700675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:27.701201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:27.701484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:27.701760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:27.702415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:27.703928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc7073-0667-44d0-810c-2758f6f1673c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:27.725665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 587/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:06:27 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:06:29.688497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b058-0f03-4878-a554-3648bfef2797 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:29.689011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b058-0f03-4878-a554-3648bfef2797 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:29.689109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b058-0f03-4878-a554-3648bfef2797 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:29.690383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 591/1178] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:29.695807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:29.696250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:29.696358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:29.714381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:29.715591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:29.781280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 588/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:30.063314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248b058-0f03-4878-a554-3648bfef2797 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:30.158239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:30.165100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248b058-0f03-4878-a554-3648bfef2797 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:30.196993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:30.197604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.197882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.198148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.208879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:30.213562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:30.214770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:06:30.215754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 592/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:30.221739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:30.281234 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:30.318815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f9a72-46fd-4b1d-92d0-8dca553eddf8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:30.352300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:30.353082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.353369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.353644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.354329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:30.354629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:30.356014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:30.364604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 589/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:30.370634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:30.468260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd0f7-3a00-481b-84ec-fbdf0cc7d88d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:30.501880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:30.502497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.502778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.503053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] PATH_INFO: `/v3/users/754df77a74ac48acb43509cc9b317f20` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.516732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] RBAC: Authorizing `identity:get_user(user_id=754df77a74ac48acb43509cc9b317f20)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:30.518180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:30.519795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 593/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/users/754df77a74ac48acb43509cc9b317f20 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:30.525983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:30.624560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905785e-ed31-466e-b356-89316c9edc84 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:30.650253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:30.659210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:30.659829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.660108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.660377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.727303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:30.728757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:30.826191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 590/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:30.832933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:30.848390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:30.937129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc27e-c6fc-4c37-bc52-fc41ae21565d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:30.970039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:30.970648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:30.970924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:30.971193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] PATH_INFO: `/v3/projects/2f74e655307943baaf13152cd669c493` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:30.972309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] RBAC: Authorizing `identity:get_project(project_id=2f74e655307943baaf13152cd669c493)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:30.973726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39c526-5be8-4a70-b434-a74512f431c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:30.975350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 594/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/projects/2f74e655307943baaf13152cd669c493 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:30.981640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:31.008974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:31.083187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4bed-bc6c-48b7-b96b-c3eadf315c32 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:31.117332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:31.117939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:31.118222 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:31.118498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:31.119250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=2f74e655307943baaf13152cd669c493, user.id=754df77a74ac48acb43509cc9b317f20) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:31.119559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:31.120981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21e7a6-8d90-47a0-8693-18f89673a5ce None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:31.126854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 591/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:06:30 2022] GET /identity/v3/role_assignments?user.id=754df77a74ac48acb43509cc9b317f20&scope.project.id=2f74e655307943baaf13152cd669c493&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:33.021975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9ef7-0bee-4f36-bc3a-09d69eb508a3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.022505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9ef7-0bee-4f36-bc3a-09d69eb508a3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.022505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9ef7-0bee-4f36-bc3a-09d69eb508a3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.023882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 595/1186] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:33.029247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.029674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.029800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.048728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:33.049909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:33.114880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 592/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:06:33.378229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9ef7-0bee-4f36-bc3a-09d69eb508a3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:33.459329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:33.477869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9ef7-0bee-4f36-bc3a-09d69eb508a3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:33.512220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:33.512838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.513118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.513491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.524916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:33.529585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:33.530673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:06:33.532576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 596/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:33.537278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:33.558366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:33.633849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785049e-cf99-431f-a1a8-ea26b74ade08 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:33.667418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:33.668021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.668304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.668578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.669235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:33.669537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:33.670898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:33.679281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 593/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:33.685108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:33.785074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f682a14-8559-46a2-b4a7-6f04d6a8b7ef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:33.818162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:33.818777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.819054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.819352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] PATH_INFO: `/v3/users/754df77a74ac48acb43509cc9b317f20` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.820453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] RBAC: Authorizing `identity:get_user(user_id=754df77a74ac48acb43509cc9b317f20)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:33.821875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:33.823454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 597/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:33 2022] GET /identity/v3/users/754df77a74ac48acb43509cc9b317f20 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:33.829265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:33.912128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:33.930073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd677c-d95a-43bd-b455-78846ebbe3fa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:33.962438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:33.963154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:33.963367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:33.963644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] PATH_INFO: `/v3/projects/2f74e655307943baaf13152cd669c493` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:33.964751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] RBAC: Authorizing `identity:get_project(project_id=2f74e655307943baaf13152cd669c493)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:33.966212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e5105-5366-43f1-b2b1-508ce8fc0496 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:33.967858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 594/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:33 2022] GET /identity/v3/projects/2f74e655307943baaf13152cd669c493 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:33.974504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:33.986971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:34.074315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4deca9-c678-4004-8810-55c71be9098c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:34.108634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:34.109286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:34.109641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:34.110178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] PATH_INFO: `/v3/projects/2f74e655307943baaf13152cd669c493/users/754df77a74ac48acb43509cc9b317f20/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:34.112471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f74e655307943baaf13152cd669c493, user_id=754df77a74ac48acb43509cc9b317f20, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:34.113925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ed51c-9983-4814-bc7e-e266d766ea76 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:34.126715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 598/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:06:33 2022] PUT /identity/v3/projects/2f74e655307943baaf13152cd669c493/users/754df77a74ac48acb43509cc9b317f20/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:06:36.025242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766727-4a21-4d06-9b2d-3cdbc7e54ef5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.025806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766727-4a21-4d06-9b2d-3cdbc7e54ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.025806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766727-4a21-4d06-9b2d-3cdbc7e54ef5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:36.027116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 595/1193] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:06:36.032715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.033105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.033274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:36.051675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:06:36.052857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:06:36.114738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:36.120120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 599/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:06:36.390800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9766727-4a21-4d06-9b2d-3cdbc7e54ef5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:36.491297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9766727-4a21-4d06-9b2d-3cdbc7e54ef5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:36.524198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:36.524810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.525080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.525347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:36.535554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:06:36.540189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:06:36.541278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:06:36.542121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 596/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:06:36.547408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:36.648992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837729-6e02-4c36-8107-e6d12e80c727 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:36.690449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:36.691056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.691369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.691647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:36.692313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:36.692615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:36.694003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:36.702581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 600/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:36.708446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:36.722854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:36.809844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b68f3-ca9b-476f-b2d9-d462e5579ad9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:36.842295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:36.842913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.843199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.843510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] PATH_INFO: `/v3/users/754df77a74ac48acb43509cc9b317f20` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:36.845167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] RBAC: Authorizing `identity:get_user(user_id=754df77a74ac48acb43509cc9b317f20)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:36.846582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:36.848197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 597/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 30 16:06:36 2022] GET /identity/v3/users/754df77a74ac48acb43509cc9b317f20 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:36.854139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:36.917469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:36.955843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e66d8-4342-4811-bb24-8ad0cd7c8a84 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:36.994816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:36.994816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:36.994816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:36.994816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:37.061709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:37.063188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:37.121785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:37.161382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 601/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:06:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:06:37.168007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:37.243691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:37.264876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356b03-89d0-446f-a3af-ea5d2a64417b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:37.300133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:37.300766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:37.301044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:37.301432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] PATH_INFO: `/v3/projects/2f74e655307943baaf13152cd669c493` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:37.305409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] RBAC: Authorizing `identity:get_project(project_id=2f74e655307943baaf13152cd669c493)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:37.305409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d841ac-7e81-49c8-9ef3-5807a694a286 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:37.305686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 598/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 30 16:06:37 2022] GET /identity/v3/projects/2f74e655307943baaf13152cd669c493 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:37.312797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:06:37.410404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97726a9-2283-49d0-a364-0b2f79c14503 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:06:37.443365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:06:37.443892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:06:37.444169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:06:37.444439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:06:37.445148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=2f74e655307943baaf13152cd669c493, user.id=754df77a74ac48acb43509cc9b317f20) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:06:37.445453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:06:37.446876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d1f05-e188-4ee7-8500-7e536bd1fdec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:06:37.452433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 602/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:06:37 2022] GET /identity/v3/role_assignments?user.id=754df77a74ac48acb43509cc9b317f20&scope.project.id=2f74e655307943baaf13152cd669c493&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:06:47.293115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=27 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:06:47.449922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:32.869286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b6cd-0f60-4ce6-9e7a-a05474b5584f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:32.869885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b6cd-0f60-4ce6-9e7a-a05474b5584f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:32.869990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b6cd-0f60-4ce6-9e7a-a05474b5584f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:32.871714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 599/1201] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:32.879260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:32.879260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:32.879260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:32.901643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:32.902772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:33.002734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 603/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:07:33.303404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b6cd-0f60-4ce6-9e7a-a05474b5584f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:33.443437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b6cd-0f60-4ce6-9e7a-a05474b5584f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:33.498309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:33.499112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:33.499551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:33.499831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:33.514866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:33.516520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:33.536510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:33.537683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:33.538737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:33.538737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:33.538737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:33.538737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 600/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:33.544991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:33.694502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744a2b-6bd8-4dd0-bfb3-d98a7f7b3a20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:33.735945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:33.744420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:33.745566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:33.746189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:33.746702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:33.747707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:33.748021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:33.751135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6e240-fb1f-45a2-9cc1-094aa450931e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:33.767932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 604/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:33.776361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:33.843724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:33.901769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286872d-0599-4935-9d1c-79e38ab1f9cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:33.947096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:33.947819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:33.948068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:33.948341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:33.948986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:33.950403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff768269-9073-4c63-8d26-769b2a4083fc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:33.983490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 601/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 30 16:07:33 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:07:36.107838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c143f3-e502-4e80-8070-e076874ef99c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:36.108439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c143f3-e502-4e80-8070-e076874ef99c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:36.108439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c143f3-e502-4e80-8070-e076874ef99c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:36.109938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 605/1206] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:36.115617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:36.116111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:36.116222 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:36.136008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:36.137198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:36.220858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 602/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:07:36.505410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c143f3-e502-4e80-8070-e076874ef99c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:36.644086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c143f3-e502-4e80-8070-e076874ef99c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:36.684741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:36.685681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:36.686149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:36.686537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:36.699242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:07:36.704197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:36.705357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:36.707051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 606/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:07:36.711571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:36.768363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:36.845062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243ef64-d9dd-45d7-9c92-2ed681e3e9a4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:36.886584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:36.887394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:36.887910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:36.888754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:36.889787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:36.890231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:36.892465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:36.905123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 603/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:36.911688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:36.973751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:37.029481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe618ce-2476-4e5d-91b6-8c6d68c082d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:37.078166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:37.079050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:37.079549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:37.079849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:37.098560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:37.100234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:37.119538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:37.121342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:37.123181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:37.123181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:37.123181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:37.127520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 607/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:37.135036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:37.261763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:37.265381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a56-0069-4e7a-8ece-7b653d2b01cb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:37.309970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:37.310675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:37.311071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:37.311477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:37.312315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:37.312734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:37.314190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:37.327797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 604/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:37.333877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:37.456643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd8a1-8c40-444d-971e-ade3c18da86d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:37.458350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:37.497975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:37.498855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:37.499227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:37.499573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:37.508442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:37.510688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:07:37.522467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:37.523791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:37.524961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:37.527423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 608/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:37.530536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:37.673888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892eb0-02b8-4881-8934-a4f4fc972d08 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:37.723186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:37.723930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:37.724277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:37.724622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:37.725555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:37.725984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:37.728346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:37.751351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 605/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:07:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:37.757601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:37.880014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69f23-9d74-43a5-910a-bc7945a7e0e2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:37.924830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:37.925450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:37.925695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:37.925978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:37.964599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:38.014695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:38.016336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:38.141363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 609/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:07:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 16:07:38.147435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:38.175837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:38.278318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18e0c-7e24-45a6-aa3c-48de089b0ec7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:38.320053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:38.320895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:38.321196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:38.321539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:38.334323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:38.336037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:38.353671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:38.354711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:38.355679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:38.355679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:38.355679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:38.359596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 606/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:38.362840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:38.439866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:38.490607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357cb-0a9e-44aa-b67f-bea690c2bd74 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:38.531163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:38.532130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:38.532591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:38.533096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:38.533890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:38.534393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:38.536122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:38.550793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 610/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:38.556701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:38.644930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:38.680623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9c102-b01b-4c7f-bbd4-a340ac0b727b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:38.719895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:38.720505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:38.720820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:38.721160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:38.728866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:38.730302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:07:38.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:38.751935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:38.753041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 607/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:38.754743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:38.875559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9757ce-1c09-4331-8b1d-f739567b8468 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:38.908742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:38.926208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:38.926936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:38.927327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:38.927658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:38.928660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:38.929161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:38.931493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241549d-9c99-4d15-b5cc-31af0e625454 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:38.948970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 611/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:07:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:38.954906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:39.082220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b2d30-dae1-45dc-abbf-17dce89b0531 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:39.116121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:39.126140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:39.126740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:39.127134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:39.127524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:39.128587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=4c81e08e950648be89483f26f546a7de) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:39.129029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:39.130747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba685a40-f9f2-457b-9654-acf43973fdaf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:39.137803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 608/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:07:38 2022] GET /identity/v3/role_assignments?user.id=4c81e08e950648be89483f26f546a7de&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:41.225539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab694bb7-1ae9-4100-a73b-6f7f9bd64d7e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.226117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab694bb7-1ae9-4100-a73b-6f7f9bd64d7e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.226117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab694bb7-1ae9-4100-a73b-6f7f9bd64d7e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.227456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 612/1220] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:41.233408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.233958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.233958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.252804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:41.254042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:41.332820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 609/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:07:41.624919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab694bb7-1ae9-4100-a73b-6f7f9bd64d7e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:41.742218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab694bb7-1ae9-4100-a73b-6f7f9bd64d7e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:41.782100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:41.782815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.783204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.783548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.798090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:07:41.803742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:41.804892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:41.806152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:41.810053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 613/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:07:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:41.811074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:41.916189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa321326-2e8d-4296-acad-cae5928a6ba1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:41.949375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:41.961684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:41.962290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:41.962570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:41.962843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:41.963561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:41.963866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:41.965230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:41.973777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 610/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:07:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:41.980552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.031039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:42.091773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884727-c3bd-43ee-a4af-a3666c85743c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.133197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.133925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.134208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.134507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.148722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.150787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:42.172252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:42.174526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:42.176719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:42.176719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:42.176719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:42.176719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 614/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:42.181688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.299910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690704-2d32-4827-9815-d2fd5944ba13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.336561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.337166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.337446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.337721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.338360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:42.338667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.340073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.353851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 611/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:42.360968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.476248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:42.484899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321cd55-a2e4-45ac-8f2c-1b562f18ad2e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.523498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.524129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.524408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.524675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.534059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.536023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:07:42.549843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:42.551205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:42.552540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 615/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:42.557035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.608618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:42.670942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e645c0d-b13c-467a-81a7-5ae570c93a49 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.710162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.710766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.711152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.711427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.712051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:42.712353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.713907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.737864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 612/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:07:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:42.745617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:42.871076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82de6ea-179c-44f0-8493-9a8a9589684c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:42.910607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:42.911270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:42.911558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:42.911855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:42.927531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:42.929018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:42.949344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:42.950681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:42.951766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:42.951766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:42.951766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:42.951766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 616/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:07:42.957080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:43.066710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b5348-2519-4e68-9e11-225f3c342888 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:43.069021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:43.112935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:43.113634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:43.114019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:43.114435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:43.115412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:43.115774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:43.117546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:43.136918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 613/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:43.142873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:43.152405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:43.260680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a258d9-1682-493f-a2d7-41102090eb19 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:43.301578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:43.302196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:43.302578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:43.302849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:43.311529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:43.313150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:07:43.325665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:43.326874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:43.327924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 617/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:07:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:43.333276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:43.445069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8803653-b7c8-4e26-ad97-712b3f360965 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:43.487572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:43.488230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:43.488518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:43.488795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:43.489496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:43.489804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:43.491575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390145d5-1210-46ed-a02e-547cbb978685 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:43.506140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 614/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:43.512718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:43.585368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:43.624233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced5a1-e2a5-4fe1-a1ed-848737a798d3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:43.662836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:43.663681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:43.664009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:43.664310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/4c81e08e950648be89483f26f546a7de/roles/77568f1330e6436e9b6db39e6cf17b82` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:43.680851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=4c81e08e950648be89483f26f546a7de, role_id=77568f1330e6436e9b6db39e6cf17b82)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:43.682345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e5f82-1ec1-4810-ad8c-c6bc1ee8deef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:43.695950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 618/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:07:43 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/4c81e08e950648be89483f26f546a7de/roles/77568f1330e6436e9b6db39e6cf17b82 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:07:45.684033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6e7c3-2fbb-486e-b530-bc1fa4839e25 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:45.684637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6e7c3-2fbb-486e-b530-bc1fa4839e25 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:45.684838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6e7c3-2fbb-486e-b530-bc1fa4839e25 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:45.686728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 615/1233] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:07:45.693380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:45.693837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:45.693980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:45.718981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:45.720564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:45.792837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 619/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:07:46.125808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6e7c3-2fbb-486e-b530-bc1fa4839e25 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:46.154415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:46.240274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6e7c3-2fbb-486e-b530-bc1fa4839e25 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:46.283261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:46.283942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:46.284240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:46.284659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:46.298054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:07:46.303046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:46.304340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 16:07:46.305392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:46.309468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 616/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 30 16:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:46.310617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:46.362778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:46.427382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad544f-524f-45e5-8b4a-510763ea8022 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:46.473018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:46.473636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:46.473915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:46.474189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:46.474868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:46.475300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:46.476574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:46.488174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 620/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 30 16:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:46.497123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:46.596981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:46.617541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784445a-c6da-4366-95c9-b4eb8ced69e0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:46.660716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:46.661560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:46.661845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:46.662197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:46.675159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:46.676773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:46.691926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:46.693412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:46.694370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:46.694370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:46.694370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:46.694370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 617/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:46.699467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:46.810404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff6416-51ce-4e7d-969a-cd1e34ec6ace None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:46.819873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:46.852055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:46.852756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:46.853049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:46.853330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:46.854065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:46.854391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:46.856005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:46.870496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 621/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:46.876644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:46.998256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6936-7545-4efc-9289-08425ef02f55 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:47.036011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:47.036773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:47.037150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:47.037467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:47.041852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:47.048356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:47.050161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:07:47.062962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:47.064209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:47.065363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 618/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:47.070429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:47.180908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58385e-4791-4c18-b403-d1e468e20ece None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:47.218630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:47.219245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:47.219533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:47.219807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:47.220538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:47.220842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:47.222268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:47.242445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 622/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:07:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:47.248635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:47.358420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fda6-4cc4-4ec4-a4fb-92e4ea70eddc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:47.396459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:47.397069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:47.397347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:47.397641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:47.466448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:47.473357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:47.474898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:47.588533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 619/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:07:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:07:47.594601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:47.632400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:47.718682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204451-587f-42f6-bc27-6ff625fac2e7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:47.756744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:47.757372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:47.757652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:47.757924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:47.768516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:47.769980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:47.785651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:47.787356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:47.788533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:47.788533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:47.788533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:47.788533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 623/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:47.793478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:47.898243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:47.905790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebc9f-408b-45f1-a88f-b396d2c7484f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:47.944654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:47.945440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:47.945745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:47.946152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:47.946898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:47.947315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:47.948853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:47.963690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 620/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:47.970134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:48.070697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:48.099865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ce2dc-04bd-40e5-ba7b-5d8c052f5fb8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:48.142587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:48.143250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:48.143533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:48.143836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:48.150944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:48.152365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:07:48.163746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:48.165063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:48.166131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 624/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:48.171559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:48.291660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34689-e56c-4ae1-bfbb-1edd808bf062 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:48.332987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:48.333592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:48.333859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:48.334130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:48.334818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:48.335120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:48.336580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d97d7-3419-4a19-a714-795fffb688bf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:48.354169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 621/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:48.361771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:48.490682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95615669-348b-4677-8e53-7b2129416dc1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:48.534377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:48.535001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:48.535381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:48.535689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:48.536611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] RBAC: Adding query filter params (role.id=77568f1330e6436e9b6db39e6cf17b82, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=4c81e08e950648be89483f26f546a7de) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:48.536874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:48.538625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6f701-c4d2-4e7d-94cc-37e318db2bb0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:48.555737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 625/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:07:48 2022] GET /identity/v3/role_assignments?user.id=4c81e08e950648be89483f26f546a7de&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=77568f1330e6436e9b6db39e6cf17b82 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:50.852282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fd59e-b90a-485e-8c24-9b57a2539f46 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:50.852809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fd59e-b90a-485e-8c24-9b57a2539f46 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:50.852809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fd59e-b90a-485e-8c24-9b57a2539f46 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:50.854639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 622/1247] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:50.862476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:50.862476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:50.862476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:50.869808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:50.885174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:50.886579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:50.976808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 626/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:07:51.290008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fd59e-b90a-485e-8c24-9b57a2539f46 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:51.315992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:51.431866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fd59e-b90a-485e-8c24-9b57a2539f46 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:51.472240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:51.472869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:51.473153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:51.473422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:51.485458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:07:51.490669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:51.491857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:51.492867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 623/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:51.499121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:51.621829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d588a5-70e5-4e0f-8692-d05ea53c7ce0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:51.656712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:51.662591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:51.663201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:51.663556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:51.663771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:51.664457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:51.664748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:51.666138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:51.674883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 627/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:51.681489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:51.773537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:51.800372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827b619-4e6b-4c2e-91b2-a163c5213cbb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:51.834480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:51.835104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:51.835412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:51.835685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:51.846874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:51.848375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:51.865618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:51.867175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:51.868310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:51.868310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:51.868310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:51.868310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 624/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:51.873433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:52.008387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714f83-1bcd-4188-9270-07eb2751997b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:52.045001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:52.045606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:52.045903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:52.046152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:52.046791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:52.047096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:52.048493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:52.060982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 628/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:52.066648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:52.181894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962724b5-d232-4b4f-bd92-461549b4da4c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:52.210580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:52.219435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:52.220062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:52.220337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:52.220633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:52.229217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:52.230702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:07:52.242488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:52.243785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:52.245281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:52.247727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 625/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:52.250186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:52.323658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:52.371251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76d5d0-a23d-4e19-8b36-9556afd67ea6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:52.409972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:52.410576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:52.410881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:52.411273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:52.412064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:52.412348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:52.414031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:52.434940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 629/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:07:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:52.440975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:52.559619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2c638-c27b-446f-bcdb-94e19ad8d1c7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:52.598843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:52.599591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:52.599949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:52.600234 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:52.670837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:52.682350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:52.683935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:52.798501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 626/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:07:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:07:52.804188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:52.943938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591d33-29df-4d2d-aeab-7de95c4a7703 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:52.962442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:52.987777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:52.988409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:52.988690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:52.988963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:52.999940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:53.001380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:53.018028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:53.019204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:53.020280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:53.020280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:53.020280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:53.020280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 630/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 16:07:53.025245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:53.129691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:53.144295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8005c7-d2c3-4b8b-adc0-c586629daafe None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:53.182731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:53.183569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:53.183857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:53.184118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:53.185293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:53.185602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:53.186964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:53.202277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 627/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:53.209108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:53.337920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31d86-897a-44a7-a6d1-6fed2c25a91c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:53.376351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:53.376975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:53.377329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:53.377601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:53.384847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:53.386312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:07:53.401217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:53.402210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:53.403863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:07:53.407907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 631/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:53.409926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:53.520203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e5aed-b9fb-4361-8739-19015a71c4cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:53.557314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:53.557931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:53.558213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:53.558486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:53.559188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:53.559525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:53.560968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7fe6c-48cb-42f4-8c0d-6c8b4b3f9429 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:53.574115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 628/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:53.581253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:53.609302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:53.691654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb07c7-a88e-496b-bbc2-4bd13927ec65 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:53.727900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:53.728523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:53.728839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:53.729118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:53.729882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=4c81e08e950648be89483f26f546a7de) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:53.730179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:53.731712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91095e37-7294-46a1-b45d-e9cf23ddb98d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:53.738270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 632/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:07:53 2022] GET /identity/v3/role_assignments?user.id=4c81e08e950648be89483f26f546a7de&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:55.656177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bb6e-9371-4e4e-823e-f9ece135893b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:55.656603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bb6e-9371-4e4e-823e-f9ece135893b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:55.656735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bb6e-9371-4e4e-823e-f9ece135893b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:55.658087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 629/1261] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:55.663406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:55.663843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:55.663960 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:55.682363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:55.683568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:55.749473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 633/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 16:07:56.020858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851bb6e-9371-4e4e-823e-f9ece135893b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.032298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.119829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851bb6e-9371-4e4e-823e-f9ece135893b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.152689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.153836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.154113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.154382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.164131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:07:56.168932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:56.170067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:56.170949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 630/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:56.176004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.194555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.274135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6185f-f197-40b5-b8ab-257c73c6b08d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.306366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.306956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.307267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.307544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.308243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:56.308546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:56.310472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:56.318336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 634/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:07:56.324102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.390555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.423747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c8cc-0cb7-43d8-99ff-7ee1af78e18f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.456307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.456914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.457191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.457460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.467685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:56.469098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:56.483602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:56.484859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:56.485773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:56.485773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:56.485773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:56.485773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 631/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:56.490736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.572591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.592421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9fcc-cba7-4396-baa7-81457009cc81 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.626466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.627068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.627388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.627668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.628312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:56.628620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:56.629983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:56.641728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 635/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:56.647449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.745323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416f1f-de6c-4cdb-8969-74d6ce330ffc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.765122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.779695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.780314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.780597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.780879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.788366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:56.789790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:07:56.801266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:56.802472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:56.803571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 632/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:56.808181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:56.912974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8299-be74-4cd4-bce5-7d63c0be50a6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:56.958655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:56.959282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:56.959567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:56.959843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:56.960572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:56.960882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:56.962312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:56.971553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:56.980328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 636/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:56.986999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:57.084263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e293a7-7840-42bd-a618-d13b5e927db6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:57.117895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:57.118509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:57.118788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:57.119062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:57.129326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:57.130746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:07:57.145096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:57.146305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:07:57.147400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:07:57.147400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:07:57.147400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:57.147400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 633/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:57.152224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:57.250586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a863d1-e874-41bb-a019-d344719f9e0d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:57.286880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:57.287516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:57.287799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:57.288075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:57.288711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:57.289015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:57.290392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:57.301955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 637/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:57.307787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:57.344189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:57.416978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b123-4b16-4ce7-860f-5bb5e7dabcef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:57.451164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:57.451949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:57.452370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:57.452650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:57.459618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:57.461073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:07:57.472535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:07:57.474328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:07:57.476594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 634/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:07:57.481555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:57.539481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:57.580734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088dd11e-806b-474a-a5ed-314119ae226e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:57.614260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:57.615059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:57.615386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:57.615663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:57.616368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:07:57.616676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:57.618154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc24647-5e80-4888-b094-e12fd0b39c85 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:57.631473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 638/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:07:57.637831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:07:57.732610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:07:57.741942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed21976-8ea5-427b-82c1-f3c3fa90abf5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:07:57.774441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:07:57.775094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:57.775409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:57.775684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] PATH_INFO: `/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/4c81e08e950648be89483f26f546a7de/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:57.777563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f6f22dea344c4b87d4ee93a282f8ef, user_id=4c81e08e950648be89483f26f546a7de, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:07:57.779540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90700e9f-84b0-4d9b-91c9-f4e2dda110e2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:07:57.792081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 635/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 30 16:07:57 2022] PUT /identity/v3/projects/b2f6f22dea344c4b87d4ee93a282f8ef/users/4c81e08e950648be89483f26f546a7de/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:07:59.669200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2fe7-71f3-4a72-b9d8-c88fe7337b20 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:59.669698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2fe7-71f3-4a72-b9d8-c88fe7337b20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:59.669799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2fe7-71f3-4a72-b9d8-c88fe7337b20 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:59.671084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 639/1274] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:07:59.676377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:07:59.676802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:07:59.676919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:07:59.694726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:07:59.695947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:07:59.762856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 636/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:07:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:08:00.027478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f2fe7-71f3-4a72-b9d8-c88fe7337b20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.127400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f2fe7-71f3-4a72-b9d8-c88fe7337b20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.144076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:00.160512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.161131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.161409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.161679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.172116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 16:08:00.176827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:00.177907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:00.178954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 640/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:00.183986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.262508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:00.280202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ea32f-fe2c-4023-91b0-b6f2a7ffddb6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.325812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.326417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.326696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.326969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.327662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:00.327967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:00.329361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:00.342726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 637/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:00.348506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.449500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232592bf-c0a9-446f-97b7-d0ba4af539cb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.483074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.483751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.484034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.484306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.495528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:00.497431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:08:00.512616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:00.513762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:08:00.514889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:08:00.514889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:08:00.514889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:00.514889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 641/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:00.519656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.617463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456b3a-32a2-4aab-80f5-6052fa738fb1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.652643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.653753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.654035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.654310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.654951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:00.655284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:00.656656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:00.669722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 638/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:00.675464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.706301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:00.774804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8334e-016a-4424-98d8-7b8ae0c72ad1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.808319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.808938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.809248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.809526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.816856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:00.818295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 16:08:00.829306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:00.831030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:00.832948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 642/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:00.837814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:00.842262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:00.939772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb691f-ec39-4e18-925b-9450d70e3ec6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:00.973562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:00.974182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:00.974471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:00.974745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:00.975500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:00.975810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:00.977229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:00.994826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 639/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:01.000644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:01.092619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:01.102846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356796-6a6d-450c-9a1c-d2a36614f447 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:01.136958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:01.138086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:01.138366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:01.138643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:01.204507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:01.205893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:01.291803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:01.302243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 643/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 30 16:08:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 16:08:01.308210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:01.368622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:01.407139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a541c3-2c81-4e9f-9d98-361a2bb7bbc4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:01.440251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:01.440872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:01.441151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:01.441423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:01.451145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:01.452604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 16:08:01.467193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:01.468315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 16:08:01.469231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 16:08:01.469231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 16:08:01.469231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:08:01.469231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 640/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:01.474081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:01.571511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f219de-1e8a-4170-acde-a31ddd3fb91a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:01.604668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:01.605261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:01.605542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:01.605813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:01.607035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:01.607369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:01.608737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:01.620106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 644/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 30 16:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:01.625821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:01.724826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eab4d-fa33-4ac7-8f2d-3c90603f722e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:01.738202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:01.758116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:01.758732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:01.759034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:01.759314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:01.766291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:01.767765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 16:08:01.779201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:01.780241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 30 16:08:01.781248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 641/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 30 16:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:01.786174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:01.835914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:01.888778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7bc1b-1361-439f-9d72-87b69dd24808 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:01.924994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:01.925588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:01.925872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:01.926147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:01.926833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:01.927140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:01.928589 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1908-a7ad-48dd-a305-49e4fba05f3a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:01.940438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 645/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 30 16:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:01.946466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:02.044183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d036873-0227-46c7-a5e3-b2f09a060fde None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:02.076794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:02.077382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:02.077664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:02.077937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:02.078651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] RBAC: Adding query filter params (role.id=8375c8e5926547ccbdfc1e86457e975a, scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef, user.id=4c81e08e950648be89483f26f546a7de) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:02.078959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:02.080399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9081b-4951-4f98-9e04-a5a4dd34b68e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:02.085992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 642/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 30 16:08:01 2022] GET /identity/v3/role_assignments?user.id=4c81e08e950648be89483f26f546a7de&scope.project.id=b2f6f22dea344c4b87d4ee93a282f8ef&role.id=8375c8e5926547ccbdfc1e86457e975a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:04.012855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddc6a-3ee8-4ee2-b5ff-7c872c4c8d68 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:04.013388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddc6a-3ee8-4ee2-b5ff-7c872c4c8d68 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:04.013388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddc6a-3ee8-4ee2-b5ff-7c872c4c8d68 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:04.014731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 646/1288] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:08:04.019913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:04.020296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:04.020466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:04.039069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:04.040338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:04.068900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:04.106196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 643/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:08:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:08:04.370094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dddc6a-3ee8-4ee2-b5ff-7c872c4c8d68 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:04.470266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dddc6a-3ee8-4ee2-b5ff-7c872c4c8d68 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:04.482059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:04.506322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:04.506949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:04.507251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:04.507537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:04.508118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:04.509489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:04.518224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:04.519521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:04.520556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 647/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:08:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:04.525315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:04.623438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d48902-c23b-4e1b-80db-fa62753fdd65 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:04.658454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:04.659053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:04.659375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:04.659652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:04.660280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:04.660584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:04.661949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b88f5f-2d08-4429-866b-bb440441dbf4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:04.667011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 644/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 30 16:08:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:08:04.674047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:04.786745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fbbd4-2f2f-46ce-a7ad-f6855bfe1175 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:04.820315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:04.820920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:04.821203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:04.821475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:04.822105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:04.822413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:04.823828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa0d05-62d7-42eb-b365-a0fa12e1e402 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:04.829160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 648/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 30 16:08:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:06.733469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1b09-b0cf-47d0-a0d5-b0d2ad7048f8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:06.734031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1b09-b0cf-47d0-a0d5-b0d2ad7048f8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:06.734031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1b09-b0cf-47d0-a0d5-b0d2ad7048f8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:06.735379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 645/1293] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:08:06.740824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f028f-03ca-490e-9edc-9c033a53446f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:06.741230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f028f-03ca-490e-9edc-9c033a53446f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:06.741387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f028f-03ca-490e-9edc-9c033a53446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:06.759400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dc7f028f-03ca-490e-9edc-9c033a53446f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:06.760587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc7f028f-03ca-490e-9edc-9c033a53446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:06.789891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:06.827750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 649/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:08:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:08:07.117345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1b09-b0cf-47d0-a0d5-b0d2ad7048f8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:07.148537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:07.218769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1b09-b0cf-47d0-a0d5-b0d2ad7048f8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:07.252593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:07.253191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:07.253477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:07.253748 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:07.254391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:07.255785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29f45b-7e51-43e9-a020-63ded22a9458 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:07.270344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 646/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 30 16:08:07 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:08:09.202299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f2c19-f27f-4811-9817-a645c9bf414e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.202834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f2c19-f27f-4811-9817-a645c9bf414e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.202834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f2c19-f27f-4811-9817-a645c9bf414e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.204239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 650/1296] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:08:09.209404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.209829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.209952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.228343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:09.229538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:09.300336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 647/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:08:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:08:09.573571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f2c19-f27f-4811-9817-a645c9bf414e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.676351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f2c19-f27f-4811-9817-a645c9bf414e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.711617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.712245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.712529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.712804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.713385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.714772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:09.723760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:09.724919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:09.725921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 651/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:09.731163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.750276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:09.831249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa15b-ae53-4b4f-becd-0d26eb302681 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:09.865361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:09.865960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:09.866249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:09.866518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:09.867138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:09.867468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:09.868838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:09.874529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 648/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:09.880720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:09.911715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:09.997538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec99af0-7bfa-4ae8-8626-c335d03de2a8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:10.035053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:10.035685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:10.035962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:10.036235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:10.036931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] RBAC: Adding query filter params (interface=public, service_id=83f2df62c7624e8eab68d7e4b4c3acb3, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:10.037238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:10.038603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0584d-a1d0-4690-8410-3fce29d691e0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:10.044210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 652/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 30 16:08:09 2022] GET /identity/v3/endpoints?service_id=83f2df62c7624e8eab68d7e4b4c3acb3&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:10.049986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:10.130595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:10.148416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c40e5e-d19b-4347-b3d0-14fb6ca11f15 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:10.181638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:10.182236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:10.182513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:10.182779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:10.183405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:10.185491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fb0a3-8fb1-46bc-9fce-4e2f2e7df925 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:10.192514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 649/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 30 16:08:10 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:08:12.104036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc1f9-2aa8-47e4-94bc-730597683f65 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:12.104036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc1f9-2aa8-47e4-94bc-730597683f65 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:12.104644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc1f9-2aa8-47e4-94bc-730597683f65 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:12.106191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 653/1302] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:08:12.111608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:12.112027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:12.112160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:12.129822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:08:12.130993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:08:12.194818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 650/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:08:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 16:08:12.463609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc1f9-2aa8-47e4-94bc-730597683f65 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:12.558169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:12.565059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc1f9-2aa8-47e4-94bc-730597683f65 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:12.597600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:12.598218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:12.598504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:12.598780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:12.599392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:12.600773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 16:08:12.610063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 16:08:12.612659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Aug 30 16:08:12.613749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 654/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 30 16:08:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 16:08:12.617451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:12.698599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:12.722671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf63ad-1042-469c-be67-385206395f32 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:12.757123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:12.757715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:12.757999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:12.758272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:12.758895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:08:12.759199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:12.760600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65f8f8-f557-420a-bf9f-68b383dfee8e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:12.765628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 651/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 30 16:08:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:08:12.771572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:08:12.873440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5654e-c684-4d2f-98cc-5113261a62b1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:08:12.907990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:08:12.908587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:08:12.908870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:08:12.909142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:08:12.909819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:08:12.911245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28625c-ade7-4a91-a5dc-92907da28c7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:08:12.930737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 655/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:08:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:08:22.763204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=30 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:08:22.926431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:17.240283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305bcd-3762-463d-ba3c-01f1ca0f7949 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:17.240763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305bcd-3762-463d-ba3c-01f1ca0f7949 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:17.241090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305bcd-3762-463d-ba3c-01f1ca0f7949 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:17.243743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 652/1307] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:17.249343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7fceb-41a0-4718-b5f5-639369e715a3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:17.249750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7fceb-41a0-4718-b5f5-639369e715a3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:17.250046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7fceb-41a0-4718-b5f5-639369e715a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:17.379582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a4a7fceb-41a0-4718-b5f5-639369e715a3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:17.382060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a4a7fceb-41a0-4718-b5f5-639369e715a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:17.539445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 656/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:09:17.838882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:17.839167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:17.839648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:17.841281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 653/1309] 10.222.0.57 () {54 vars in 965 bytes} [Tue Aug 30 16:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:09:17.847823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3788-8331-47f3-807f-4fd5e94d622e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:17.848166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3788-8331-47f3-807f-4fd5e94d622e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:17.848390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3788-8331-47f3-807f-4fd5e94d622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:17.944136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9bac3788-8331-47f3-807f-4fd5e94d622e None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:17.945356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9bac3788-8331-47f3-807f-4fd5e94d622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:17.970913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:18.032776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 657/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Tue Aug 30 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:09:18.042158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:18.044030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:18.174432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6fc4-6e11-4923-ab40-9760b39aaa04 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:18.252985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:18.253632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:18.253913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:18.254018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:18.255635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:18.313785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:18.372936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:18.374425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f6f2d-7ca0-4cb3-9dac-7c4fcc742d46 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:18.395562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 654/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:20.617465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01afd99-32b0-417d-aeab-b53f04d24981 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:20.617977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01afd99-32b0-417d-aeab-b53f04d24981 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:20.618081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01afd99-32b0-417d-aeab-b53f04d24981 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:20.619477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 658/1312] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:20.624941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290311b-282d-46cf-a689-a9656f2b1263 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:20.625367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290311b-282d-46cf-a689-a9656f2b1263 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:20.625508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290311b-282d-46cf-a689-a9656f2b1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:20.707535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9290311b-282d-46cf-a689-a9656f2b1263 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:20.708839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9290311b-282d-46cf-a689-a9656f2b1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:20.806845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 655/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:21.110958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:21.111262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:21.111561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:21.112894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 659/1314] 10.222.0.57 () {54 vars in 965 bytes} [Tue Aug 30 16:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:21.119661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092875de-2081-47bd-a532-11e9c7bbc905 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:21.119948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092875de-2081-47bd-a532-11e9c7bbc905 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:21.120218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092875de-2081-47bd-a532-11e9c7bbc905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:21.131773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:21.197198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-092875de-2081-47bd-a532-11e9c7bbc905 None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:21.198410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-092875de-2081-47bd-a532-11e9c7bbc905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:21.273861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 656/1315] 10.222.0.57 () {58 vars in 1062 bytes} [Tue Aug 30 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:09:21.283684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:21.285271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:21.394758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8dcfb-1d40-43b5-b913-68c04be4ee1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:21.468682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:21.471528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:21.472545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:21.473173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:21.473886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:21.475626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:21.587465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:21.588926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2facd-f013-4dad-b6a7-c58f99e4cecd service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:21.609080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 660/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:24.081001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:24.081570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:24.081570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:24.082907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 657/1317] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:24.088188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e2d3-c9cb-4e34-8e9d-3259562e5a49 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:24.088612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e2d3-c9cb-4e34-8e9d-3259562e5a49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:24.088753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e2d3-c9cb-4e34-8e9d-3259562e5a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:24.161482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-63e0e2d3-c9cb-4e34-8e9d-3259562e5a49 None None] MFA Rules not processed for user `b3a51bbf235046feb98976f941a96c39`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:24.162767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-63e0e2d3-c9cb-4e34-8e9d-3259562e5a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:24.214846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:24.259879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 661/1318] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:24.543609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:24.545146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:24.650659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0dc6e-e6e0-4eb7-8b19-40595e59f8c5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:24.653530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:24.724057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:24.724664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:24.724944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:24.725217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:24.726797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:24.834374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:24.837027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16595d98-dcb4-42fd-bec9-929934a9c784 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:24.857602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 658/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:27.364087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:27.364624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:27.364624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:27.365998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 662/1320] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:27.372251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d84ac-43bb-4c92-a671-b12d2fcdafd4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:27.372251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d84ac-43bb-4c92-a671-b12d2fcdafd4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:27.372808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d84ac-43bb-4c92-a671-b12d2fcdafd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:27.412807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:27.438788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f60d84ac-43bb-4c92-a671-b12d2fcdafd4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:27.440031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f60d84ac-43bb-4c92-a671-b12d2fcdafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:27.535125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 659/1321] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:27.822921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:27.824512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:27.933697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef0e66-e8dc-4acd-8cc8-caff1e84ce28 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:28.007022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:28.007647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:28.007931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:28.008204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:28.009822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:28.018223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:28.124594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:28.126049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc25e8-f56e-49ec-a329-404037e73395 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:28.144258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 663/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:30.731078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:30.731692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:30.731692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:30.733019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 660/1323] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:30.738459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d8d3-28cb-4c80-8028-7a0067af0533 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:30.738896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d8d3-28cb-4c80-8028-7a0067af0533 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:30.739001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d8d3-28cb-4c80-8028-7a0067af0533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:30.805269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-33e5d8d3-28cb-4c80-8028-7a0067af0533 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:30.807110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-33e5d8d3-28cb-4c80-8028-7a0067af0533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:30.865015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:30.912070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 664/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:31.203807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:31.205369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:31.314122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b5843-c651-478e-8575-a532dece9bc3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:31.327215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:31.391493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:31.391990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:31.392351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:31.392575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:31.394100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:31.508933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:31.510371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab5081-6b2b-4329-b440-7b77cc6a0e1c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:31.528741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 661/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:33.513441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:33.513970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:33.513970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:33.515261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 665/1326] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:33.520568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf6cb-6744-486a-982c-99ec84305953 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:33.521048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf6cb-6744-486a-982c-99ec84305953 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:33.521157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf6cb-6744-486a-982c-99ec84305953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:33.572228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:33.590217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ba3bf6cb-6744-486a-982c-99ec84305953 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:33.591454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ba3bf6cb-6744-486a-982c-99ec84305953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:33.692108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 662/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:33.969884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:33.971587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:34.079137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec52659-7b42-4e1e-bf00-3449baa2df83 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:34.160162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:34.160861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:34.161211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:34.161559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:34.163279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:34.181207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:34.280456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:34.281913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53124-6144-4653-bc89-1a751ed810eb service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:34.301381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 666/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:36.266769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:36.267341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:36.267441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:36.268861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 663/1329] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:09:36.274627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1226d-ee72-4f35-8103-c73449d05097 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:36.275118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1226d-ee72-4f35-8103-c73449d05097 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:36.275249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1226d-ee72-4f35-8103-c73449d05097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:36.346396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dbe1226d-ee72-4f35-8103-c73449d05097 None None] MFA Rules not processed for user `b3a51bbf235046feb98976f941a96c39`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:36.347657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dbe1226d-ee72-4f35-8103-c73449d05097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:36.420743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:36.465701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 667/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:36.755011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:36.756628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:36.873673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e748c-b5d5-4cc1-a95c-e8e21d3df459 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:36.894530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:36.949350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:36.949957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:36.950236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:36.950509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:36.952137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:37.060513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:37.061858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e806b9-1d4a-4d6b-85bb-2943c436849d service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:37.080433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 664/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:39.242902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:39.243510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:39.243510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:39.244825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 668/1332] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:09:39.249876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8673c-f32d-4952-a211-c938110c7ea5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:39.250282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8673c-f32d-4952-a211-c938110c7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:39.250429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8673c-f32d-4952-a211-c938110c7ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:39.314488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:39.317227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8ef8673c-f32d-4952-a211-c938110c7ea5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:39.318952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8ef8673c-f32d-4952-a211-c938110c7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:39.416395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 665/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:39.698017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:39.699580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:39.803809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a748a-33df-4731-955c-9ec9be9c72f5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:39.878461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:39.879064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:39.879381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:39.879654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:39.881267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:39.908863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:39.990838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:39.992315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8688-6596-4183-af5a-887e1431d424 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:40.010889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 669/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:41.969786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:41.970349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:41.970349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:41.971750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 666/1335] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:41.976753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9b16-0166-4880-8bc4-9f58b0c54fa9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:41.977194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9b16-0166-4880-8bc4-9f58b0c54fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:41.977312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9b16-0166-4880-8bc4-9f58b0c54fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:42.044536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-927d9b16-0166-4880-8bc4-9f58b0c54fa9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:42.045753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-927d9b16-0166-4880-8bc4-9f58b0c54fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:42.120506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:42.143700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 670/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:09:42.430536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:42.432120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:42.535689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553abc9-2e9f-4aca-9fb0-1f55452b3241 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:42.564774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:42.610434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:42.611321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:42.611604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:42.611880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:42.613470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:42.732043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:42.733425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ae03-62e9-40ee-9753-b19c27312acc service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:42.752695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 667/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:44.991488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:44.992047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:44.992047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:44.994028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 671/1338] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:44.999057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e298-eb40-4646-bd5c-d713b16f695c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:44.999496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e298-eb40-4646-bd5c-d713b16f695c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:44.999640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523e298-eb40-4646-bd5c-d713b16f695c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:45.064226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d523e298-eb40-4646-bd5c-d713b16f695c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:45.065423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d523e298-eb40-4646-bd5c-d713b16f695c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:45.073601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:45.160438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 668/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:45.439980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:45.441518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:45.553220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28c223-b73d-421f-a4c6-d83f42d7d6bd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:45.626849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:45.627485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:45.627769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:45.628044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:45.629632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:45.665972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:45.740224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:45.741600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14546304-6f9f-443a-83da-a1ab0280fda1 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:45.760005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 672/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:49.683467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:49.684003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:49.684003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:49.685362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 669/1341] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:49.690479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c677fd-7bbd-435b-b735-1a1105e872e8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:49.690906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c677fd-7bbd-435b-b735-1a1105e872e8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:49.691034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c677fd-7bbd-435b-b735-1a1105e872e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:49.760941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c5c677fd-7bbd-435b-b735-1a1105e872e8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:49.762149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5c677fd-7bbd-435b-b735-1a1105e872e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:49.852254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:49.864880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 673/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:50.148192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:50.149712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:50.257453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791aafa-41ea-4e43-bcda-a5c443d7409a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:50.295896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:50.333021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:50.333619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:50.333895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:50.334165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:50.335801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:50.447152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:50.448602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c291a75-4ca3-475c-bb48-91e00135efde service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:50.468127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 670/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:53.079638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:53.080197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:53.080197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:53.081549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 674/1344] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:53.087180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1abd4-211b-498d-b742-e1f1be59f4fd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:53.087704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1abd4-211b-498d-b742-e1f1be59f4fd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:53.087820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1abd4-211b-498d-b742-e1f1be59f4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:53.156553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b4c1abd4-211b-498d-b742-e1f1be59f4fd None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:53.157761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b4c1abd4-211b-498d-b742-e1f1be59f4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:53.176034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:53.264211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 671/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:53.544691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:53.546222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:53.657492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39873f-c1cf-4ab6-8245-8244ba5a5012 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:53.738959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:53.739592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:53.739871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:53.740141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:53.741711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:53.788226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:53.856469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:53.857937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c5a57-f153-470b-8659-2d784a4ee1df service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:53.877436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 675/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:09:57.834767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:57.835332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:57.835426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:57.836899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 672/1347] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:09:57.842622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde363a-db79-439e-a04e-e15c426f0ecc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:57.843108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde363a-db79-439e-a04e-e15c426f0ecc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:57.843202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde363a-db79-439e-a04e-e15c426f0ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:57.918564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-acde363a-db79-439e-a04e-e15c426f0ecc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:09:57.919933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-acde363a-db79-439e-a04e-e15c426f0ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:58.022152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:58.025302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 676/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:09:58.312261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:09:58.313786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:58.428658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3645f-7f2f-47ab-9fdb-b8df67608e69 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:09:58.478051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:09:58.507036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:09:58.507697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:09:58.508647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:09:58.508647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:09:58.509920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:09:58.629967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:09:58.631453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a046c-b7aa-4a9c-9818-f77bd0bed565 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:09:58.653919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 673/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:03.633431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df21d-7f6a-443f-b8d2-5654447e5beb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:03.633876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df21d-7f6a-443f-b8d2-5654447e5beb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:03.633876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df21d-7f6a-443f-b8d2-5654447e5beb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:03.635470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 677/1350] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:03.643887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa22e-2e18-417a-87e0-824e80911db7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:03.643887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa22e-2e18-417a-87e0-824e80911db7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:03.643887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aa22e-2e18-417a-87e0-824e80911db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:03.753883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-588aa22e-2e18-417a-87e0-824e80911db7 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:03.755113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-588aa22e-2e18-417a-87e0-824e80911db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:03.795732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:03.922444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 674/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:10:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:08.177737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f4f4a-a0c5-414d-a3bc-f87f3d869870 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:08.178384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f4f4a-a0c5-414d-a3bc-f87f3d869870 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:08.178620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f4f4a-a0c5-414d-a3bc-f87f3d869870 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:08.180763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 678/1352] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:08.186495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676da5-f256-43d5-8ea2-78f5e8a8b7a1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:08.187003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676da5-f256-43d5-8ea2-78f5e8a8b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:08.187086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676da5-f256-43d5-8ea2-78f5e8a8b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:08.269814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f9676da5-f256-43d5-8ea2-78f5e8a8b7a1 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:08.271426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f9676da5-f256-43d5-8ea2-78f5e8a8b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:08.346426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:08.390445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 675/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:10:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:09.704504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67a86b-8e94-4b83-a5a6-f4086ccfcd04 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:09.705032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67a86b-8e94-4b83-a5a6-f4086ccfcd04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:09.705032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67a86b-8e94-4b83-a5a6-f4086ccfcd04 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:09.706375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 679/1354] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:09.712468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7e83-75a1-4575-bcf9-bfc7aacdbebb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:09.712962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7e83-75a1-4575-bcf9-bfc7aacdbebb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:09.713056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7e83-75a1-4575-bcf9-bfc7aacdbebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:09.792790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c24c7e83-75a1-4575-bcf9-bfc7aacdbebb None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:09.793999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c24c7e83-75a1-4575-bcf9-bfc7aacdbebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:09.914469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 676/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:10:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:13.332067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0cd0-1dd1-49c6-b8c6-e727e31eb52e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:13.332648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0cd0-1dd1-49c6-b8c6-e727e31eb52e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:13.332648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0cd0-1dd1-49c6-b8c6-e727e31eb52e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:13.333971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 680/1356] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:13.339548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638e7d3-789f-4bb4-8719-bffec50329fb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:13.340008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638e7d3-789f-4bb4-8719-bffec50329fb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:13.340128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638e7d3-789f-4bb4-8719-bffec50329fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:13.384360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:13.422088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8638e7d3-789f-4bb4-8719-bffec50329fb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:13.424186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8638e7d3-789f-4bb4-8719-bffec50329fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:13.542347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 677/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:13.869450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:13.869917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:13.870335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:13.872151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 681/1358] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 30 16:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:13.882268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479391a-9816-4e10-8876-1af57086435b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:13.882575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479391a-9816-4e10-8876-1af57086435b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:13.882829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479391a-9816-4e10-8876-1af57086435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:13.986641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b479391a-9816-4e10-8876-1af57086435b None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:13.988221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b479391a-9816-4e10-8876-1af57086435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:14.068570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:14.142501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 678/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 30 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:14.150595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:14.153790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:14.310437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee9a8f-ca94-4f04-9116-9a0a15f8c116 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:14.392094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:14.436533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:14.437603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:14.438125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:14.438608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:14.440566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:14.538445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:14.540093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc0c1e-3a1a-4779-8c2b-ba667dc7b6ed service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:14.559020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 682/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:16.715884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:16.716451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:16.716537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:16.718886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 679/1361] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:10:16.724633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c260d9b-c6b2-475d-9e8b-2c13cc907c16 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:16.725599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c260d9b-c6b2-475d-9e8b-2c13cc907c16 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:16.725887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c260d9b-c6b2-475d-9e8b-2c13cc907c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:16.803771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7c260d9b-c6b2-475d-9e8b-2c13cc907c16 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:16.805010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7c260d9b-c6b2-475d-9e8b-2c13cc907c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:16.848864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:16.924197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 683/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:17.230003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:17.231943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:17.394150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bad71-9088-4f63-a67a-e8fcc30d7be2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:17.404934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:17.537539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:17.538508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:17.538795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:17.539068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:17.540706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:17.640323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:17.641901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b9288-9540-4394-b8a5-de9a2974620c service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:17.661696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 680/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:19.931662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257465dd-49d5-4b5f-ad4d-52a8ed622fb3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:19.932563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257465dd-49d5-4b5f-ad4d-52a8ed622fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:19.932563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257465dd-49d5-4b5f-ad4d-52a8ed622fb3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:19.933753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 684/1364] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:19.939505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1b9a-fa5a-48eb-af89-7ef5c5908046 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:19.940381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1b9a-fa5a-48eb-af89-7ef5c5908046 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:19.940861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1b9a-fa5a-48eb-af89-7ef5c5908046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:19.958370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:20.015320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-245f1b9a-fa5a-48eb-af89-7ef5c5908046 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:20.016520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-245f1b9a-fa5a-48eb-af89-7ef5c5908046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:20.123554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 681/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:20.413063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:20.413424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:20.413805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:20.415406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 685/1366] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 30 16:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:20.421302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2df1-4fbd-48f4-bc85-641ed8159ac2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:20.421589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2df1-4fbd-48f4-bc85-641ed8159ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:20.421862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2df1-4fbd-48f4-bc85-641ed8159ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:20.505742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-405c2df1-4fbd-48f4-bc85-641ed8159ac2 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:20.507559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-405c2df1-4fbd-48f4-bc85-641ed8159ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:20.544662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:20.638458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 682/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 30 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:20.647747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:20.649451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:20.819469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371752cf-a543-4339-a39e-57ab04482a84 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:20.841931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:20.958187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:20.958861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:20.959112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:20.959415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:20.961763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:21.065665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:21.067331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1d40c-b35b-4c55-99fb-f9def4ab1b1e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:21.088923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 686/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:21.641046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7050-6bde-446c-8c5f-1e609b02e15e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:21.641616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7050-6bde-446c-8c5f-1e609b02e15e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:21.641616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7050-6bde-446c-8c5f-1e609b02e15e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:21.642939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 683/1369] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:21.648519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24489383-74a0-4b87-812a-0e8c82da9f98 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:21.648807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24489383-74a0-4b87-812a-0e8c82da9f98 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:21.649084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24489383-74a0-4b87-812a-0e8c82da9f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:21.683150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:21.720299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24489383-74a0-4b87-812a-0e8c82da9f98 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:21.721505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24489383-74a0-4b87-812a-0e8c82da9f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:21.828436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 687/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:22.413131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:22.413131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:22.413131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:22.415671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 684/1371] 10.222.0.57 () {54 vars in 964 bytes} [Tue Aug 30 16:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:22.418599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57664b-743e-4953-a4ce-3bab6ef36aea None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:22.419095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57664b-743e-4953-a4ce-3bab6ef36aea None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:22.419310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57664b-743e-4953-a4ce-3bab6ef36aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:22.508175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fb57664b-743e-4953-a4ce-3bab6ef36aea None None] MFA Rules not processed for user `de5581201518429191f74cf8af19fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:22.509771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb57664b-743e-4953-a4ce-3bab6ef36aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:22.560763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:22.592000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 688/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Aug 30 16:10:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:22.599468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:22.601554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:22.729984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b1876-5ae1-4fe2-b1f1-147924e0599f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:22.796226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:22.842388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:22.843260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:22.843468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:22.844029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:22.845971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:23.033665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:23.034680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cf6af-2d0a-4bbd-afc0-38830cb23717 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:23.069298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:23.069626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:23.070154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:23.072305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 685/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:23.075507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 689/1374] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:23.076688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985b15-860f-4d0d-8aa4-66efe3159488 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:23.076974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985b15-860f-4d0d-8aa4-66efe3159488 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:23.077258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985b15-860f-4d0d-8aa4-66efe3159488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:23.164414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-77985b15-860f-4d0d-8aa4-66efe3159488 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:23.165759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-77985b15-860f-4d0d-8aa4-66efe3159488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:23.187355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:23.273387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 686/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:10:23.559766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:23.561284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:23.716698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3360d-a06e-465b-afad-9d2959da3b3a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:23.772736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:23.833909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:23.835052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:23.835363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:23.835635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:23.837205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:23.926750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:23.928426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528ac7-35b6-48f3-8d45-6fa80aa1936b service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:23.947637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 690/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:25.928011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:25.928526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:25.928526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:25.930062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 687/1377] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:25.935427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae84f-5509-4ec2-94fb-098e9ba8eb61 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:25.935863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae84f-5509-4ec2-94fb-098e9ba8eb61 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:25.935991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae84f-5509-4ec2-94fb-098e9ba8eb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:26.005121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b73ae84f-5509-4ec2-94fb-098e9ba8eb61 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:26.006888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b73ae84f-5509-4ec2-94fb-098e9ba8eb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:26.013913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:26.125049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 691/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:26.422802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:26.424529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:26.604729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d44a7-9af2-49dc-87cb-0600125af3de None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:26.613785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:26.752440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:26.752994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:26.753271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:26.753568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:26.755102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:26.862125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:26.863980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f48e-b573-4ba0-a8a4-9867076c42ca service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:26.893500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 688/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:28.897983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:28.898516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:28.898516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:28.900001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 692/1380] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:28.905110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce66c6-7d33-4dca-bf3e-0fde1e480112 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:28.905607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce66c6-7d33-4dca-bf3e-0fde1e480112 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:28.905689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce66c6-7d33-4dca-bf3e-0fde1e480112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:28.917827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:28.982651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b5ce66c6-7d33-4dca-bf3e-0fde1e480112 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:28.983951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5ce66c6-7d33-4dca-bf3e-0fde1e480112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:29.099269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 689/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:10:29.392736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:29.394368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:29.586588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:29.598767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c6e6-6c7a-4aab-92ce-7ae41eed3040 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:29.739760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:29.740889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:29.741179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:29.741458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:29.743125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:29.842380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:29.844073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9d60-b5c0-4fda-8249-b3418ea20416 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:29.855068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:29.876373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 693/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:31.832179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:31.832921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:31.832921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:31.834128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 690/1383] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:10:31.839406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942344-d4a2-43e1-8353-08c872ef4a05 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:31.839863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942344-d4a2-43e1-8353-08c872ef4a05 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:31.839969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942344-d4a2-43e1-8353-08c872ef4a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:31.933731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ac942344-d4a2-43e1-8353-08c872ef4a05 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:31.934958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ac942344-d4a2-43e1-8353-08c872ef4a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:32.038936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 694/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:32.337202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:32.338786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:32.428406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:32.508849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e73bf-53c8-40b7-a8c5-d7c90f1dbf8e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:32.657815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:32.658475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:32.658763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:32.659043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:32.660867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:32.713011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:32.755294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:32.757454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16815296-0cf1-4d23-80e9-e3528af063b9 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:32.775884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 691/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:34.937790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:34.938550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:34.938550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:34.940287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 695/1386] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:34.945797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f3bc5-16b7-4c12-b93e-0863bcc723c9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:34.946483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f3bc5-16b7-4c12-b93e-0863bcc723c9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:34.946483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f3bc5-16b7-4c12-b93e-0863bcc723c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:35.022062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9d5f3bc5-16b7-4c12-b93e-0863bcc723c9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:35.023404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9d5f3bc5-16b7-4c12-b93e-0863bcc723c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:35.124065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 692/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:35.416847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:35.418362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:35.498934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:35.571117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777f9d-21d9-42c4-ae06-2de0a7e45ffe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:35.693263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:35.694783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:35.695061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:35.695363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:35.696949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:35.734814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:35.790929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:35.792600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022a5db-d3d4-476f-ac22-693f2e089767 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:35.812561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 696/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:37.767036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9a6c-49d0-42e3-8b27-a3e8caef75c8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:37.767648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9a6c-49d0-42e3-8b27-a3e8caef75c8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:37.767648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9a6c-49d0-42e3-8b27-a3e8caef75c8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:37.769014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 693/1389] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:37.774437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14731d-3d21-4bdb-bd1b-84ddcb23abe4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:37.774883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14731d-3d21-4bdb-bd1b-84ddcb23abe4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:37.775005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14731d-3d21-4bdb-bd1b-84ddcb23abe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:37.842262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6e14731d-3d21-4bdb-bd1b-84ddcb23abe4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:37.843497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6e14731d-3d21-4bdb-bd1b-84ddcb23abe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:37.947992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 697/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:38.178959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:38.179311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:38.179540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:38.180889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 694/1391] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:38.186121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da913bf2-713d-47fa-af5f-c6b8789ef678 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:38.186438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da913bf2-713d-47fa-af5f-c6b8789ef678 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:38.186699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da913bf2-713d-47fa-af5f-c6b8789ef678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:38.202492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:38.246391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:38.248057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:38.256665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-da913bf2-713d-47fa-af5f-c6b8789ef678 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:38.257839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-da913bf2-713d-47fa-af5f-c6b8789ef678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:38.279278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:38.356533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 698/1392] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:38.399979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f063c-79bf-4ce3-8fa2-d52f24e4a817 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:38.507415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:38.519602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:38.520207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:38.520489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:38.520760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:38.522336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:38.611889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:38.613478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77961466-7b0a-4d91-a97a-455ec23be93f service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:38.633676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 695/1393] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:38.678249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:38.678547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:38.678827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:38.680506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 699/1394] 10.222.0.57 () {54 vars in 967 bytes} [Tue Aug 30 16:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:38.685855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8b39-15e7-4985-9372-79775f8bacbe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:38.686147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8b39-15e7-4985-9372-79775f8bacbe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:38.686425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8b39-15e7-4985-9372-79775f8bacbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:38.778864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c53c8b39-15e7-4985-9372-79775f8bacbe None None] MFA Rules not processed for user `c34f7dd7c12a4262a6ace2f07b6475c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:38.780075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c53c8b39-15e7-4985-9372-79775f8bacbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:38.838624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:38.864159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 696/1395] 10.222.0.57 () {58 vars in 1064 bytes} [Tue Aug 30 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:38.872341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:38.873898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:38.979534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:38.987761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdf97e-5f6f-44f5-bb73-29279b08df38 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:39.062733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:39.063275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:39.063558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:39.063827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:39.065417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:39.176581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:39.178090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adeb482-3700-41a5-8f6b-c04663811eb2 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:39.196497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 700/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:10:39.946313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8e01-d3f9-4a39-8174-1c58433c88b9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:39.946791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8e01-d3f9-4a39-8174-1c58433c88b9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:39.946882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b8e01-d3f9-4a39-8174-1c58433c88b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.015363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2e6b8e01-d3f9-4a39-8174-1c58433c88b9 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:40.016584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e6b8e01-d3f9-4a39-8174-1c58433c88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:40.105793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 697/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:10:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:40.149017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:40.149494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:40.149713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.151149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 701/1398] 10.222.0.57 () {54 vars in 943 bytes} [Tue Aug 30 16:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:40.157653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da95f-b3f9-4b67-b334-31de02b789bc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:40.158043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da95f-b3f9-4b67-b334-31de02b789bc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:40.158394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da95f-b3f9-4b67-b334-31de02b789bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.236878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f55da95f-b3f9-4b67-b334-31de02b789bc None None] MFA Rules not processed for user `058f62fc005a41b08ee30fb42192e31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:40.238011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f55da95f-b3f9-4b67-b334-31de02b789bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:40.251944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:40.354729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 698/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Tue Aug 30 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:40.371705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:40.376298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:40.409271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:40.528323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578abb-bff1-4f09-8529-6571917ee524 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:40.628136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4eca3-9ef7-46d0-a2e8-0906f988b344 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:40.628427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4eca3-9ef7-46d0-a2e8-0906f988b344 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:40.628706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4eca3-9ef7-46d0-a2e8-0906f988b344 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.630051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 699/1400] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:40.634814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:40.635010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025972-b2cf-438e-a3f7-881293f02ce5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:40.635250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025972-b2cf-438e-a3f7-881293f02ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:40.635453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:40.635565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025972-b2cf-438e-a3f7-881293f02ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.635737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:40.636010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:40.638303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:40.647832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:40.704850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-56025972-b2cf-438e-a3f7-881293f02ce5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:40.706073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-56025972-b2cf-438e-a3f7-881293f02ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:40.728856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:40.730535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:40.738971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 702/1401] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:10:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:10:40.789783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:40.813576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 700/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:10:41.112809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:41.112809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:41.213982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310bb95-05a0-4424-bc39-7e6aba62c473 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:41.281354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:41.348185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:41.348731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:41.349217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:41.349488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:41.351073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:41.452524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:41.454129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18359d1-8a66-43f7-95f4-74069482af18 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:41.472762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 703/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:43.456911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:43.457492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:43.457598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:43.459113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 701/1404] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:43.464182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0858d-d825-4215-9005-6b1f28e1ee0a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:43.464971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0858d-d825-4215-9005-6b1f28e1ee0a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:43.465251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0858d-d825-4215-9005-6b1f28e1ee0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:43.544328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fdf0858d-d825-4215-9005-6b1f28e1ee0a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:43.545811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fdf0858d-d825-4215-9005-6b1f28e1ee0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:43.564496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:43.677003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 704/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:43.986646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:43.988492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:44.197099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14f6a-f72b-4041-b83f-2e83cec73534 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:44.273136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:44.349794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:44.350404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:44.350689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:44.350964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:44.352618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:44.458696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:44.460279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694759-554a-4269-a503-9ee93399aef5 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:44.480859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 702/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:46.478504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:46.479096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:46.479096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:46.480411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 705/1407] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:10:46.488919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fc051-2a14-4780-9dfc-19ec37df951d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:46.489368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fc051-2a14-4780-9dfc-19ec37df951d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:46.489753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fc051-2a14-4780-9dfc-19ec37df951d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:46.566823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-179fc051-2a14-4780-9dfc-19ec37df951d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:10:46.569317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-179fc051-2a14-4780-9dfc-19ec37df951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:46.590306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:46.686085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 703/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:10:46.978671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:10:46.981942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:47.153935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:47.156426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdf96a-f9f1-4976-8d30-a64944ae46c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:10:47.293275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:10:47.293903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:10:47.294278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:10:47.294746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:10:47.296387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:10:47.409108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:10:47.410877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d57f87-c90a-4a56-97ad-dc3baa94eb4d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:10:47.438177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 706/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:10:56.683812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=35 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:10:57.435431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:02.931636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3465c6-73e9-495f-961e-1ede65f47bcb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.932270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3465c6-73e9-495f-961e-1ede65f47bcb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.932270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3465c6-73e9-495f-961e-1ede65f47bcb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:02.933534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 704/1410] 10.222.0.57 () {54 vars in 918 bytes} [Tue Aug 30 16:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:02.938926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e95a-4316-4b1b-8742-e72f454b677d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:02.939405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e95a-4316-4b1b-8742-e72f454b677d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:02.939504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e95a-4316-4b1b-8742-e72f454b677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:03.016688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aaa1e95a-4316-4b1b-8742-e72f454b677d None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:03.017896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aaa1e95a-4316-4b1b-8742-e72f454b677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:03.126393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 707/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 16:11:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:03.140716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdc972-c523-4474-a54d-661c3c29583b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:03.140998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdc972-c523-4474-a54d-661c3c29583b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:03.141273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdc972-c523-4474-a54d-661c3c29583b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:03.142605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 705/1412] 10.222.0.57 () {54 vars in 918 bytes} [Tue Aug 30 16:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:03.147748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad457e-0cd1-4d5b-9ec2-03e2ac843214 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:03.148040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad457e-0cd1-4d5b-9ec2-03e2ac843214 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:03.148323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad457e-0cd1-4d5b-9ec2-03e2ac843214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:03.227394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:03.230584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c9ad457e-0cd1-4d5b-9ec2-03e2ac843214 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:03.232501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c9ad457e-0cd1-4d5b-9ec2-03e2ac843214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:03.349555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 708/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 16:11:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:05.737349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:05.738181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:05.738181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:05.739278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:05.744978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c423318-f5b3-4372-9424-bdc12a421046 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:05.745401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c423318-f5b3-4372-9424-bdc12a421046 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:05.745532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c423318-f5b3-4372-9424-bdc12a421046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:05.813452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9c423318-f5b3-4372-9424-bdc12a421046 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:05.814695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9c423318-f5b3-4372-9424-bdc12a421046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:05.871392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:05.915808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:06.209621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:06.211202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:06.364339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37875-c06d-400b-a0f7-2ff579f8909c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:06.438085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:06.479578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:06.480109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:06.480393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:06.480695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:06.482279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:06.572692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:06.574305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a210-5b9e-4417-a218-3a70200707ac service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:06.592637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 707/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:07.488003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.488555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.488555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.490492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:07.495486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8ce4-75df-4c96-9c84-61fe5e48b9ce None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:07.495940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8ce4-75df-4c96-9c84-61fe5e48b9ce None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:07.496116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8ce4-75df-4c96-9c84-61fe5e48b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:07.571715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3f7a8ce4-75df-4c96-9c84-61fe5e48b9ce None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:07.572909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3f7a8ce4-75df-4c96-9c84-61fe5e48b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.599416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:07.670970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:07.819832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:07.821264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:07.970311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5d64-69e6-4aae-aca6-4a861092de33 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:08.010843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:08.084594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:08.085110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:08.085395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:08.085669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:08.087288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:08.179946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:08.181574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2bed9-ff3d-4957-9120-8dfb8bbcbc0f service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:08.200287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 711/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:09.086162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:09.087053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:09.087244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:09.088758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:09.093752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3abf1b-a6f7-4f3b-86e9-07953fc4055b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:09.094184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3abf1b-a6f7-4f3b-86e9-07953fc4055b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:09.094300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3abf1b-a6f7-4f3b-86e9-07953fc4055b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:09.150195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:09.172784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ad3abf1b-a6f7-4f3b-86e9-07953fc4055b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:09.173983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ad3abf1b-a6f7-4f3b-86e9-07953fc4055b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:09.275770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:09.429979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:09.431628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:09.584911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92192373-4e5e-423a-8d6a-cb5af7d3f539 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:09.597511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:09.723002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:09.723556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:09.724374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:09.724589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:09.726192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:09.822487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:09.824130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a6a1d-86dd-4316-b860-a9dae66b24f9 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:09.842546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 710/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:10.721403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:10.721979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:10.721979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:10.723372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:10.729130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535843f-8e1c-4764-83ba-9229cf6832f7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:10.729606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535843f-8e1c-4764-83ba-9229cf6832f7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:10.729728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535843f-8e1c-4764-83ba-9229cf6832f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:10.749703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:10.807056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4535843f-8e1c-4764-83ba-9229cf6832f7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:10.808297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4535843f-8e1c-4764-83ba-9229cf6832f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:10.907188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:11.060706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:11.062908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:11.184862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:11.223417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed06353-6511-468d-95bc-f2a4f0660f66 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:11.341586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:11.342193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:11.342470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:11.342739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:11.344391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:11.420342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:11.434719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:11.436421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03784bf0-6ddf-47c8-849e-42252b2eb262 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:11.454712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 714/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:12.322059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:12.322646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:12.322646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:12.323992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:12.328887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e138a5-4a9e-429f-ad0c-9bda3dbcf67e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:12.329289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e138a5-4a9e-429f-ad0c-9bda3dbcf67e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:12.329446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e138a5-4a9e-429f-ad0c-9bda3dbcf67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:12.409276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-57e138a5-4a9e-429f-ad0c-9bda3dbcf67e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:12.410484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-57e138a5-4a9e-429f-ad0c-9bda3dbcf67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:12.508785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:12.656839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:12.658335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:12.741535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:12.802792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d88dd-80c8-4eb4-8337-cc5342d9ffa7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:12.915889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:12.916496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:12.916776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:12.917047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:12.918667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:12.967803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:13.010847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:13.012477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed8f9d-56f6-4039-999f-bc3394ca4398 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:13.030185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 713/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:13.845337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:13.845871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:13.846010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:13.847295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:13.852466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fccdc-d82d-4f36-bb74-13f2d47de86b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:13.852921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fccdc-d82d-4f36-bb74-13f2d47de86b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:13.853022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fccdc-d82d-4f36-bb74-13f2d47de86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:13.937131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e31fccdc-d82d-4f36-bb74-13f2d47de86b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:13.938373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e31fccdc-d82d-4f36-bb74-13f2d47de86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:14.037338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:14.191529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:14.193562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:14.245354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:14.344806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84291b9-4da8-4ab3-b85c-f7de8e332a76 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:14.456574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:14.457089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:14.457366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:14.457635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:14.459292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:14.466202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:14.554200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:14.555852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ff6d-0271-4df6-9902-fe04936ff272 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:14.574133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 717/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:15.391592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:15.392201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:15.392201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:15.393497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:15.399054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5257-e8dc-4b43-8965-c102d13e9340 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:15.399544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5257-e8dc-4b43-8965-c102d13e9340 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:15.399661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5257-e8dc-4b43-8965-c102d13e9340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:15.476091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-20cc5257-e8dc-4b43-8965-c102d13e9340 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:15.477286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-20cc5257-e8dc-4b43-8965-c102d13e9340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:15.546703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:15.583048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:15.734161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:15.735716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:15.753598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:15.897229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecac47-47b1-4e6b-b37b-be4c3138df36 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:15.992841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:16.016801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:16.017405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:16.017687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:16.017975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:16.019617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:16.110086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:16.111994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350497-9d31-40d8-b70e-2ef22d2ccd23 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:16.129687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 716/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:16.943444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:16.944048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:16.944048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:16.945424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:16.950448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff7e5-060b-40fb-a5bd-665ec445bbb1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:16.950905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff7e5-060b-40fb-a5bd-665ec445bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:16.951014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff7e5-060b-40fb-a5bd-665ec445bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:17.027015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3bdff7e5-060b-40fb-a5bd-665ec445bbb1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:17.028250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3bdff7e5-060b-40fb-a5bd-665ec445bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:17.070467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:17.124543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:17.274403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:17.275878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:17.422865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c1186-96c3-4dbf-8b88-e664fab3cf3f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:17.477929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:17.538116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:17.538621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:17.538904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:17.539184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:17.540874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:17.636432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:17.638044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fbca1-7b95-41fa-b1c0-de073d9a6c65 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:17.656775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 720/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:18.470034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:18.470638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:18.470638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:18.471947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:18.477397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944547a-3b5c-4402-b025-da1d3e09150f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:18.477802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944547a-3b5c-4402-b025-da1d3e09150f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:18.477959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944547a-3b5c-4402-b025-da1d3e09150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:18.555873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9944547a-3b5c-4402-b025-da1d3e09150f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:18.557083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9944547a-3b5c-4402-b025-da1d3e09150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:18.559098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:18.662243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:18.813019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:18.814452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:18.968692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961cd7e-68e0-4a45-930a-0ed58b168919 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:19.004566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:19.094520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:19.095130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:19.095439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:19.095711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:19.097289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:19.186622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:19.188262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279d5bd-7644-4f21-900f-7a9d1c8df4d2 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:19.206898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 719/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:21.042934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:21.043516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:21.043516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:21.044884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:21.050255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806abd0-9dae-4ef0-8a02-4cf9ac3df7df None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:21.050794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806abd0-9dae-4ef0-8a02-4cf9ac3df7df None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:21.050794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806abd0-9dae-4ef0-8a02-4cf9ac3df7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:21.093369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:21.119574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6806abd0-9dae-4ef0-8a02-4cf9ac3df7df None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:21.120770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6806abd0-9dae-4ef0-8a02-4cf9ac3df7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:21.219384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:21.508738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:21.510280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:21.614844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf67b4c-690e-4b78-9e8e-e5163f78f1c4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:21.648098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:21.696759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:21.697364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:21.697643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:21.697918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:21.699597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:21.808868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:21.810316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb12f-89ad-44ff-93a8-bf2e0576e969 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:21.828096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 723/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:23.882479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:23.883049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:23.883049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:23.884438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:23.890005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f11e-adce-4426-b012-ddd9eebcd9a7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:23.890444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f11e-adce-4426-b012-ddd9eebcd9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:23.890585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f11e-adce-4426-b012-ddd9eebcd9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:23.957134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2730f11e-adce-4426-b012-ddd9eebcd9a7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:23.958422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2730f11e-adce-4426-b012-ddd9eebcd9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:23.971282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:24.059675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:24.347588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:24.349225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.454461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa72171-d5fa-46c8-9f95-ccf9510bc65f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:24.467174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:24.531832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:24.532451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:24.532731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:24.533003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:24.534600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:24.643533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:24.644896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c195ba5-1615-47bd-a663-31111607017f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:24.663549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 722/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:26.918908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.919472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.919592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.920852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:26.926162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63482-fd9b-472e-819e-f29d1ac97894 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:26.926611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63482-fd9b-472e-819e-f29d1ac97894 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:26.926715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63482-fd9b-472e-819e-f29d1ac97894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:26.976274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:26.997131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c3e63482-fd9b-472e-819e-f29d1ac97894 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:26.998354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3e63482-fd9b-472e-819e-f29d1ac97894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:27.093533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:27.380488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:27.382562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:27.489253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d229b1b-f449-42c5-941a-d97cf6474d08 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:27.532064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:27.567241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:27.567866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:27.568453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:27.568736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:27.570343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:27.681050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:27.682494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42029e-072a-447a-aab4-87667cfbdc0e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:27.700277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 726/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:29.743014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:29.743631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:29.743631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:29.744955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:11:29.750287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d1e4-6b8a-46f5-965e-3b4257131c0c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:29.750713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d1e4-6b8a-46f5-965e-3b4257131c0c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:29.750875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d1e4-6b8a-46f5-965e-3b4257131c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:29.817912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3dc9d1e4-6b8a-46f5-965e-3b4257131c0c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:29.819190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3dc9d1e4-6b8a-46f5-965e-3b4257131c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:29.839663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:29.917836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:30.205392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:30.206903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:30.310977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20bd12-7b59-4bbe-9d33-f4b97954c1db None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:30.330276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:30.384941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:30.385540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:30.385822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:30.386095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:30.387728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:30.497832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:30.499345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75f585-2927-408e-b9a8-55dc13442d1c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:30.518153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 725/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:32.497641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:32.498205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:32.498205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:32.499630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:32.504937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccdf7-4188-49b4-9220-cfdf252c5e42 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:32.505424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccdf7-4188-49b4-9220-cfdf252c5e42 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:32.505521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccdf7-4188-49b4-9220-cfdf252c5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:32.569487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:32.572330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cabccdf7-4188-49b4-9220-cfdf252c5e42 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:32.574084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cabccdf7-4188-49b4-9220-cfdf252c5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:32.670702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:32.951389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:32.952923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:33.057394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f35189-117e-4b24-bdba-16872b8e0d41 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:33.106694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:33.132214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:33.132816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:33.133095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:33.133366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:33.134984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:33.246224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:33.247966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71e96f-e5fe-4d13-a6de-8bfd95f25ca8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:33.268110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 729/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:35.515246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:35.515781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:35.515781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:35.517150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:35.522892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b65-a1b0-48c7-bd7d-945aac73b08e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:35.523418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b65-a1b0-48c7-bd7d-945aac73b08e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:35.523539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b65-a1b0-48c7-bd7d-945aac73b08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:35.589344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-942d2b65-a1b0-48c7-bd7d-945aac73b08e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:35.590475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-942d2b65-a1b0-48c7-bd7d-945aac73b08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:35.619893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:35.689331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:35.974592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:35.976191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:36.081942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3816d-8ac2-4b39-b7da-471054c696e5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:36.110373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:36.155917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:36.156512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:36.156788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:36.157056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:36.158629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:36.269031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:36.270420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb45b05-6425-410d-a03f-3ff6d3deaa09 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:36.288138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 728/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:38.252603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:38.253152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:38.253250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:38.254551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:38.260373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411803e-cf1a-43b6-a9e9-e797bf522264 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:38.260838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411803e-cf1a-43b6-a9e9-e797bf522264 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:38.260935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411803e-cf1a-43b6-a9e9-e797bf522264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:38.327603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4411803e-cf1a-43b6-a9e9-e797bf522264 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:38.328816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4411803e-cf1a-43b6-a9e9-e797bf522264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:38.337777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:38.426694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:38.708782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:38.710434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:38.815406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a9e76-339d-41fa-84fe-eb35df16255c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:38.870951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:38.889961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:38.890569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:38.890850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:38.891128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:38.892763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:39.003487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:39.004920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97e25-b8a2-4a07-9b51-16abbf3ddc4a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:39.022682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 732/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:40.979850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:40.980390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:40.980390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:40.981771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:40.987164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c644c-7e35-4130-b15a-247b7a56d915 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:40.987647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c644c-7e35-4130-b15a-247b7a56d915 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:40.987770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c644c-7e35-4130-b15a-247b7a56d915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:41.055097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-466c644c-7e35-4130-b15a-247b7a56d915 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:41.056372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-466c644c-7e35-4130-b15a-247b7a56d915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:41.097557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:41.157353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:41.446799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:41.448380 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:41.553393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458acf-4f80-4020-bf72-f4c1d4a49a42 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:41.592552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:41.629579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:41.630189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:41.630473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:41.630750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:41.632975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:41.743023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:41.744407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314abd6f-4b9b-4556-8230-50e28654b057 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:41.762600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 731/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:43.675633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:43.676146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:43.676246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:43.677545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:43.682837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c43cc-8319-48a9-93e4-2b5ea7213496 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:43.683281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c43cc-8319-48a9-93e4-2b5ea7213496 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:43.683437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c43cc-8319-48a9-93e4-2b5ea7213496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:43.749967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ed4c43cc-8319-48a9-93e4-2b5ea7213496 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:43.751203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ed4c43cc-8319-48a9-93e4-2b5ea7213496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:43.769289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:43.848524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:44.132344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:44.133866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:44.241848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35815e10-d6b9-4982-852e-a2a5015ad24f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:44.308467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:44.315618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:44.316285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:44.316563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:44.316838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:44.318955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:44.428425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:44.429876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fd593-135d-4aed-bb46-f90575c61300 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:44.448199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 735/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:46.431873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:46.432521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:46.432521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:46.433784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:11:46.438921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c0f80-dfeb-4625-845c-ae831fd68af5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:46.439377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c0f80-dfeb-4625-845c-ae831fd68af5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:46.439493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c0f80-dfeb-4625-845c-ae831fd68af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:46.517268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-684c0f80-dfeb-4625-845c-ae831fd68af5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:11:46.518485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-684c0f80-dfeb-4625-845c-ae831fd68af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:46.568053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:46.618492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:11:46.905747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:11:46.907329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:47.016512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e2eb-cddd-42b9-8cbd-c9f0f0548265 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:11:47.063479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:47.089347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:11:47.089952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:11:47.090233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:11:47.090509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:11:47.092136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:11:47.207845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:11:47.209368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19450d5b-37d1-40a6-9a2c-e96495fc9902 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:11:47.228138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 734/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:11:56.616649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=22 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:11:57.225822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=66 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:06.709339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c114826-a5cf-40f4-96e5-bd8c54ea59e2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:06.709828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c114826-a5cf-40f4-96e5-bd8c54ea59e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:06.709939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c114826-a5cf-40f4-96e5-bd8c54ea59e2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:06.711308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 737/1471] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:06.716635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8973a7-396f-4a6b-91da-34b2d5011e06 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:06.717090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8973a7-396f-4a6b-91da-34b2d5011e06 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:06.717284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8973a7-396f-4a6b-91da-34b2d5011e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:06.784243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7e8973a7-396f-4a6b-91da-34b2d5011e06 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:06.785498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e8973a7-396f-4a6b-91da-34b2d5011e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:07.001016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 735/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:07.294292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:07.294577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:07.294853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:07.296224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 738/1473] 10.222.0.57 () {54 vars in 967 bytes} [Tue Aug 30 16:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:07.302039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d41507-d5f8-4d51-a3d4-b19289fb7379 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:07.302354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d41507-d5f8-4d51-a3d4-b19289fb7379 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:07.302639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d41507-d5f8-4d51-a3d4-b19289fb7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:07.378174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d0d41507-d5f8-4d51-a3d4-b19289fb7379 None None] MFA Rules not processed for user `c34f7dd7c12a4262a6ace2f07b6475c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:07.379473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d0d41507-d5f8-4d51-a3d4-b19289fb7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:07.399845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:07.455564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 736/1474] 10.222.0.57 () {58 vars in 1064 bytes} [Tue Aug 30 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:12:07.463091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:07.464683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:07.569458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbe951-cf04-4043-ad77-cb60000bb9af None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:07.652013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:07.652615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:07.652891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:07.653158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:07.654724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:07.708741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:07.766557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:07.768055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446caa8-16e4-407d-8045-30bcaeaec812 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:07.791747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 739/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:09.789652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:09.790228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:09.790323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:09.791849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 737/1476] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:09.797308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c275-e581-4613-8566-8b31d3b1c974 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:09.797784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c275-e581-4613-8566-8b31d3b1c974 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:09.797901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c275-e581-4613-8566-8b31d3b1c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:09.864146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-35d6c275-e581-4613-8566-8b31d3b1c974 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:09.865598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35d6c275-e581-4613-8566-8b31d3b1c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:09.971713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 740/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:10.248714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:10.250165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:10.378988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2fc26-2bc1-4db1-8227-0c42fd8483d2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:10.431507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:10.477264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:10.477879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:10.478158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:10.478432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:10.479079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:10.480894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:12:10.480894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:12:10.480894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:12:10.482045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d8c1b-6269-49b8-a219-448a0da4c334 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:10.502228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 738/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 30 16:12:10 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:12:12.442714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:12.443316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:12.443316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:12.445332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 741/1479] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:12.450631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e537f-a20c-4ac3-bf7f-cd307776c0b4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:12.451060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e537f-a20c-4ac3-bf7f-cd307776c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:12.451238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e537f-a20c-4ac3-bf7f-cd307776c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:12.518941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-333e537f-a20c-4ac3-bf7f-cd307776c0b4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:12.520205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-333e537f-a20c-4ac3-bf7f-cd307776c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:12.616884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 739/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:12.905895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:12.907514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:12.918256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:13.059283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695f68b-5fe2-4111-be0a-446cb60028f1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:13.141210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:13.175280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:13.175886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:13.176171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:13.176450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:13.178050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:13.268695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:13.270340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e210e8a-79e7-4738-bd9a-da65c40a2b20 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:13.288951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 742/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:15.385197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:15.385904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:15.385904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:15.387076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 740/1482] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:15.392521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394c42-8124-4734-9b0e-a153c70df188 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:15.392959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394c42-8124-4734-9b0e-a153c70df188 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:15.393091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394c42-8124-4734-9b0e-a153c70df188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:15.459624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f394c42-8124-4734-9b0e-a153c70df188 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:15.460842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f394c42-8124-4734-9b0e-a153c70df188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:15.508732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:15.560000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 743/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:12:15.850104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:15.851768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:15.875758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:15.964049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff581-6906-40d3-930b-791ee270f6ce None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:16.039182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:16.039814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:16.040101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:16.040376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:16.041968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:16.115437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:16.153583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:16.155020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c796-16fa-4bfa-a704-723a50e7b2f4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:16.172908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 741/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:18.209255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:18.210377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:18.210377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:18.211674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 744/1485] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:18.216836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00ce9-2bd3-44e1-b3a9-8e7b9e2a2c59 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:18.217295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00ce9-2bd3-44e1-b3a9-8e7b9e2a2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:18.217406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00ce9-2bd3-44e1-b3a9-8e7b9e2a2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:18.282837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6ab00ce9-2bd3-44e1-b3a9-8e7b9e2a2c59 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:18.284105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6ab00ce9-2bd3-44e1-b3a9-8e7b9e2a2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:18.380820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 742/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:18.670532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:18.672108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:18.781975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895cc95-37e5-46dd-8944-2a575f20619d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:18.853367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:18.855392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:18.856075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:18.856558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:18.857004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:18.858788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:18.973201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:18.974684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaba019-3550-4853-9996-43b1100c71a8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:18.994401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 745/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:21.079821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:21.080396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:21.080396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:21.081723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 743/1488] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:21.087382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a47d2-74dd-4528-b4f8-a5d3852ee8a0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:21.087805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a47d2-74dd-4528-b4f8-a5d3852ee8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:21.087940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a47d2-74dd-4528-b4f8-a5d3852ee8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:21.154549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db8a47d2-74dd-4528-b4f8-a5d3852ee8a0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:21.155770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db8a47d2-74dd-4528-b4f8-a5d3852ee8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:21.207821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:21.253624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 746/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:12:21.546455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:21.548052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:21.583907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:21.653804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88748-bbfa-45b0-a94d-ea455bfaf64c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:21.728585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:21.729188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:21.729472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:21.729745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:21.731357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:21.812228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:21.842344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:21.843822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883c6e9-8f44-45e3-9d19-236c2f5516d9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:21.861584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 744/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:24.053818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:24.054390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:24.054390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:24.056052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 747/1491] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 30 16:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:12:24.061500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c32d8-8623-494c-af50-cf16d1210c33 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:24.061914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c32d8-8623-494c-af50-cf16d1210c33 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:24.062081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c32d8-8623-494c-af50-cf16d1210c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:24.133947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-329c32d8-8623-494c-af50-cf16d1210c33 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:12:24.135158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-329c32d8-8623-494c-af50-cf16d1210c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:24.231996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 745/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 30 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:12:24.521458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:12:24.523033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:24.628166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00a8c6-4508-4739-981d-dab043dc01e1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:12:24.703173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:12:24.703811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:12:24.704096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:12:24.704367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:12:24.706054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:12:24.715818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:24.819416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:12:24.820796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c69f-fe83-4933-824c-772296a84ad0 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:12:24.838962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 748/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:12:34.230147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=82 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:12:34.836681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:37.963637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:37.964242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:37.964242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.041212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:38.042432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:38.141509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 746/1494] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:38.149562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:38.150070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:38.150172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.226911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:38.228099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:38.327379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 749/1495] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:38.332162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:38.333353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:38.344532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec593c8a-2dce-414e-ab69-d5c3cc265af3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:38.406780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:38.440174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:38.440838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:38.441121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:38.441399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.442038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:13:38.442342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:38.443383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:38.443383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:13:38.443383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:13:38.444258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:38.457420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 747/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:13:38.466291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:38.475606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bec91-3ab8-4379-bf83-0157c6f45f2b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:38.540877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:38.575607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:38.576208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:38.576483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:38.576747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.577419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:38.579206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:38.579206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:13:38.579206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:13:38.580448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:38.639240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 750/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:13:38 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:13:38.643577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:38.645309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:38.679434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dc6ca-fbd3-4c32-a4a5-912ba1044326 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:38.775550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:38.776153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:38.776433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:38.776703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:38.777368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:38.778367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:38.778367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:13:38.778367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:13:38.779606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:38.820855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 748/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:13:38 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:13:38.825146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:38.826923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.011954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b525-52c5-4be9-8e23-c85285f00ab8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:39.100137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:39.112166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:39.112860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.113059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.113333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.113984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:39.115016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:39.115016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:13:39.115016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:13:39.115958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:39.129310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 751/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:13:39.134195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:39.135757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.147308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f56562-949c-4ba9-a9f9-80986fe93e11 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:39.172041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:39.242770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:39.243524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.243955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.244259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.244873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:39.245887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:39.245887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:13:39.245887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:13:39.246778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc97e5-5287-4b37-b94e-35a81d6b54bc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:39.260048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 749/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:13:39.264226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:39.265768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.298516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff0aa4-674e-4448-8e24-6848771b3f72 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:39.392658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:39.393225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.393508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.393784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] PATH_INFO: `/v3/projects/773b6090c0934ec7909fb6c9699e9f11/users/ba5b4ab9c9df4047881772f8e5abe99a/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.415870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=773b6090c0934ec7909fb6c9699e9f11, user_id=ba5b4ab9c9df4047881772f8e5abe99a, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:39.416900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:39.416900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:13:39.416900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:13:39.418843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19e32a-6972-470c-a310-0a0857739e7c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:39.445866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 752/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:13:39 2022] PUT /identity/v3/projects/773b6090c0934ec7909fb6c9699e9f11/users/ba5b4ab9c9df4047881772f8e5abe99a/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:39.457958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab708a-c5cc-4b7a-ae7b-c000ba7229b7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.458253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab708a-c5cc-4b7a-ae7b-c000ba7229b7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.458536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab708a-c5cc-4b7a-ae7b-c000ba7229b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.516528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-57ab708a-c5cc-4b7a-ae7b-c000ba7229b7 None None] MFA Rules not processed for user `ba5b4ab9c9df4047881772f8e5abe99a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:39.517841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57ab708a-c5cc-4b7a-ae7b-c000ba7229b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.603339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 750/1502] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:39.616396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.617206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.617491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.619099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 753/1503] 10.222.0.57 () {54 vars in 964 bytes} [Tue Aug 30 16:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:13:39.624375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35316a-9bd6-4088-b952-28f0ab0f7c93 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.624669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35316a-9bd6-4088-b952-28f0ab0f7c93 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.624951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35316a-9bd6-4088-b952-28f0ab0f7c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.656356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:39.701633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c35316a-9bd6-4088-b952-28f0ab0f7c93 None None] MFA Rules not processed for user `de5581201518429191f74cf8af19fb58`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:39.702823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c35316a-9bd6-4088-b952-28f0ab0f7c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.777523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 751/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Aug 30 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:39.787347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:39.789395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:39.838712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:39.894261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd1341-ca9f-4758-83cb-b4ff6362d073 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:39.970470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:39.971074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:39.971391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:39.971671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:39.973262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:40.070835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:40.072341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:40.075870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:40.090822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 754/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:13:40.135928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fba-29c2-494e-8454-07e43966edb9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:40.136359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fba-29c2-494e-8454-07e43966edb9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:40.136475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fba-29c2-494e-8454-07e43966edb9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:40.137733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 752/1506] 10.222.0.57 () {52 vars in 830 bytes} [Tue Aug 30 16:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:13:40.141387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:40.142600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:40.242448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd7db0-bfdd-44de-a4e2-92e6655e0c6c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:40.340151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:40.340854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:40.341332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:40.341612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] PATH_INFO: `/v3/users/ba5b4ab9c9df4047881772f8e5abe99a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:40.353886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5b4ab9c9df4047881772f8e5abe99a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:40.354944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:40.354944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:13:40.354944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:13:40.356195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:40.365784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:40.424842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5b4ab9c9df4047881772f8e5abe99a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:40.438421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5b4ab9c9df4047881772f8e5abe99a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:40.438979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5b4ab9c9df4047881772f8e5abe99a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:40.448513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5b4ab9c9df4047881772f8e5abe99a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:40.473556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5b4ab9c9df4047881772f8e5abe99a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:40.479202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 755/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:13:40 2022] DELETE /identity/v3/users/ba5b4ab9c9df4047881772f8e5abe99a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:40.483641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:40.483936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:40.484218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:40.543641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:13:40.544759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:40.621652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:40.634271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 753/1508] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:13:40.644599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:40.645799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:40.745829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44784c-ba04-4aa8-9fc6-0262ab021e3a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:40.800799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:40.821169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:40.821781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:40.822067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:40.822344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:40.823970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:40.935573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:40.937018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b7fe3-4d05-45a7-be7b-2698fd5d9c34 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:40.955437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 756/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:13:41.303512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:13:41.304774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:13:41.316585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c5c30-70d7-4172-ac8c-88cc236feb71 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:13:41.418073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:13:41.418704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:13:41.419022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:13:41.419334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] PATH_INFO: `/v3/projects/773b6090c0934ec7909fb6c9699e9f11` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:13:41.429271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773b6090c0934ec7909fb6c9699e9f11)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:13:41.430289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:13:41.430289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:13:41.430289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:13:41.431529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:13:41.489799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773b6090c0934ec7909fb6c9699e9f11', 'request_id': 'req-b88596b1-d516-43ce-a18f-6b8d33f81392'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:41.503199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773b6090c0934ec7909fb6c9699e9f11', 'request_id': 'req-b88596b1-d516-43ce-a18f-6b8d33f81392'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:41.509481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773b6090c0934ec7909fb6c9699e9f11', 'request_id': 'req-b88596b1-d516-43ce-a18f-6b8d33f81392'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:41.536654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] The token cache is being invalidate because project 773b6090c0934ec7909fb6c9699e9f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:13:41.536946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:41.537378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b88596b1-d516-43ce-a18f-6b8d33f81392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:13:41.539505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 754/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:13:41 2022] DELETE /identity/v3/projects/773b6090c0934ec7909fb6c9699e9f11 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:13:50.953129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=62 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:13:51.535108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:07.154410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:07.155024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:07.155024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:07.156366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:07.161594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec354-32cb-4760-9f37-708087ea95e5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:07.162047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec354-32cb-4760-9f37-708087ea95e5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:07.162205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec354-32cb-4760-9f37-708087ea95e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:07.246099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-62fec354-32cb-4760-9f37-708087ea95e5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:07.247601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-62fec354-32cb-4760-9f37-708087ea95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:07.349239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:07.504300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:07.506497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:07.625921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110b3e1-7916-4de1-a368-2e3b92ac0bed None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:07.632224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:07.632841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:07.633132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:07.633409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:07.642002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:07.643640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca119-f2aa-47ac-8773-2c580b152f78 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:07.653746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 758/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:08.491160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673586b5-1627-4a55-8b3c-421e39680dcc None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:08.491782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673586b5-1627-4a55-8b3c-421e39680dcc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:08.491782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673586b5-1627-4a55-8b3c-421e39680dcc None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:08.493165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:08.498737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79504ca-a98a-48b4-835e-5e2ad1410675 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:08.499174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79504ca-a98a-48b4-835e-5e2ad1410675 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:08.499317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79504ca-a98a-48b4-835e-5e2ad1410675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:08.519547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b79504ca-a98a-48b4-835e-5e2ad1410675 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:08.520751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b79504ca-a98a-48b4-835e-5e2ad1410675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:08.529822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:08.684816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673586b5-1627-4a55-8b3c-421e39680dcc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:08.699047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673586b5-1627-4a55-8b3c-421e39680dcc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:08.704498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:08.705085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:08.705367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:08.705642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:08.713756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:08.715376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c60d71-fbf2-46d7-a3c8-9823513fb595 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:08.724779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 757/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:09.580068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c84d2-b4b1-4b18-8204-7262ebe184e7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:09.580796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c84d2-b4b1-4b18-8204-7262ebe184e7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:09.580796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c84d2-b4b1-4b18-8204-7262ebe184e7 None None] PATH_INFO: `/v3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:09.581934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 760/1517] 10.222.0.48 () {54 vars in 925 bytes} [Tue Aug 30 16:15:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:15:09.596134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54055a9-b303-47df-a738-37f4e3a44506 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:09.596683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54055a9-b303-47df-a738-37f4e3a44506 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:09.596808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54055a9-b303-47df-a738-37f4e3a44506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:09.660879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b54055a9-b303-47df-a738-37f4e3a44506 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:09.662103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b54055a9-b303-47df-a738-37f4e3a44506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:09.679170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:09.735959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 758/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Tue Aug 30 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:13.669656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8bbdb-83ad-4479-a513-91c62020264a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:13.670164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8bbdb-83ad-4479-a513-91c62020264a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:13.670276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8bbdb-83ad-4479-a513-91c62020264a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:13.671578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 761/1519] 10.222.0.57 () {54 vars in 961 bytes} [Tue Aug 30 16:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:13.676593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1816784-467d-424d-b151-311101e89c76 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:13.676998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1816784-467d-424d-b151-311101e89c76 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:13.677159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1816784-467d-424d-b151-311101e89c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:13.696629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a1816784-467d-424d-b151-311101e89c76 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:13.697872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a1816784-467d-424d-b151-311101e89c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:13.706895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 759/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Aug 30 16:15:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:13.714310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8bbdb-83ad-4479-a513-91c62020264a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:13.728115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8bbdb-83ad-4479-a513-91c62020264a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:13.733720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:13.734313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:13.734597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:13.734873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:13.743067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:13.744689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a94c8-a442-4206-acb4-058b77d386a4 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:13.754825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 762/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:19.951114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45a37-baae-4283-a9db-45e99a710a06 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:19.951686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45a37-baae-4283-a9db-45e99a710a06 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:19.951686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45a37-baae-4283-a9db-45e99a710a06 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:19.953063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:19.957899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875347-acc8-462c-941e-a4c9ad5f1600 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:19.958309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875347-acc8-462c-941e-a4c9ad5f1600 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:19.958436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875347-acc8-462c-941e-a4c9ad5f1600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:19.978727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e8875347-acc8-462c-941e-a4c9ad5f1600 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:19.979968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8875347-acc8-462c-941e-a4c9ad5f1600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:19.988340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:20.146631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45a37-baae-4283-a9db-45e99a710a06 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:20.161204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45a37-baae-4283-a9db-45e99a710a06 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:20.166399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:20.166979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:20.167285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:20.167558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:20.176054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:20.177612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eff623-55f2-4896-9566-650b822cba20 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:20.187500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 761/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:15:21.016813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc068f86-7f09-4c76-86c0-c3fef4b5eb93 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:21.017424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc068f86-7f09-4c76-86c0-c3fef4b5eb93 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:21.017424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc068f86-7f09-4c76-86c0-c3fef4b5eb93 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:21.018756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 16:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:15:21.023851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73c1ec-fe28-4307-a930-63d9b5b3bf54 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:21.024493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73c1ec-fe28-4307-a930-63d9b5b3bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:21.024493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73c1ec-fe28-4307-a930-63d9b5b3bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:21.044006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9b73c1ec-fe28-4307-a930-63d9b5b3bf54 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:21.045218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9b73c1ec-fe28-4307-a930-63d9b5b3bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:21.053103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:21.241446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc068f86-7f09-4c76-86c0-c3fef4b5eb93 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:21.256679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc068f86-7f09-4c76-86c0-c3fef4b5eb93 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:21.262150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:21.262752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:21.263652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:21.263929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:21.272304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:21.273832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a2103-e54a-4175-a95a-d9e7bfb45e90 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:21.283189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 765/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:25.042943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:25.043527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:25.043527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:25.044880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:25.050160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2aab-1e50-4f21-87ea-4a3a5d0a168c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:25.050600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2aab-1e50-4f21-87ea-4a3a5d0a168c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:25.050710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2aab-1e50-4f21-87ea-4a3a5d0a168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:25.070311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-270a2aab-1e50-4f21-87ea-4a3a5d0a168c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:25.071467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-270a2aab-1e50-4f21-87ea-4a3a5d0a168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:25.080133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:25.412376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:25.414091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:25.493639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7a838-c63a-4a31-b4d7-19af847c4932 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:25.497482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:25.498079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:25.498363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:25.498694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:25.507679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:25.509102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2db63-4754-4cd4-886a-5e5b5bb1a5f6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:25.518375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 764/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:15:27.414409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774275-6bf9-4086-9d7e-0785337bb714 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:27.414996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774275-6bf9-4086-9d7e-0785337bb714 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:27.414996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774275-6bf9-4086-9d7e-0785337bb714 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:27.416377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:27.421722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bac6de-e0aa-487b-8fc6-023b9ed9db91 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:27.422170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bac6de-e0aa-487b-8fc6-023b9ed9db91 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:27.422273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bac6de-e0aa-487b-8fc6-023b9ed9db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:27.441581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-06bac6de-e0aa-487b-8fc6-023b9ed9db91 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:27.442767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-06bac6de-e0aa-487b-8fc6-023b9ed9db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:27.450453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:27.736511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e774275-6bf9-4086-9d7e-0785337bb714 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:27.747508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e774275-6bf9-4086-9d7e-0785337bb714 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:27.751678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:27.752265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:27.752539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:27.752812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:27.761551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:27.762968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f7eba-c5cc-476c-977a-31f265f51339 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:27.772943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 768/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:31.165033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:31.165633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:31.165633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:31.166902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:31.172193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e59e9-2131-4590-81f6-a0b88f184ff6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:31.172665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e59e9-2131-4590-81f6-a0b88f184ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:31.172782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e59e9-2131-4590-81f6-a0b88f184ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:31.180416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:31.193332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e08e59e9-2131-4590-81f6-a0b88f184ff6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:31.194515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e08e59e9-2131-4590-81f6-a0b88f184ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:31.202771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:31.761051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:31.762750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:31.828642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:31.842532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59728851-7ad3-4e01-8915-366cf779f7db None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:31.846218 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:31.846824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:31.847107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:31.847416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:31.856304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:31.858348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e9303-c505-4174-bca3-d7bcc2e1a8c7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:31.867796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 767/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:33.821446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:33.821986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:33.821986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:33.823374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 16:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:15:33.828530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc195a-0dc1-4636-9b63-b93a076c4e98 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:33.828986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc195a-0dc1-4636-9b63-b93a076c4e98 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:33.829085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc195a-0dc1-4636-9b63-b93a076c4e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:33.850876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-00fc195a-0dc1-4636-9b63-b93a076c4e98 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:15:33.852109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00fc195a-0dc1-4636-9b63-b93a076c4e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:33.860447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:15:34.419434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:15:34.421160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:15:34.434128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b1bea-392e-433d-b7a0-6ea7e3fbbac6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:15:34.438180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:15:34.438768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:15:34.439047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:15:34.439365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:15:34.448238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:15:34.449663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d942-cd44-4b77-af05-d10d0601f021 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:15:34.460063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 771/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:15:43.845957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=11 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:15:44.455611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:00.097228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.097930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.097930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.132735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.134126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.144492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 769/1540] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:00.153786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.154382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.154382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.184276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.185576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.199987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 772/1541] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.205546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.221025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec0e1-0a38-4855-a447-f800c3498204 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.229334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.230144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.230438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.230852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.231563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:00.231869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.233830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bef22-71c6-430d-899f-18911ed5472d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.254564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 770/1542] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:00.265405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.275730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1904c-2314-4acf-9854-0bd01ae3d0ea None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.302088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.302744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.303016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.303317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.304046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.306432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ea0e2-b767-4dad-a1b6-a3e7a6e3ad6a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.308636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.310035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.310489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.354504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.356407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.378012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 771/1543] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:00.379623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 773/1544] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:00.384649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.388055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.388361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.388620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.407201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a708a-1f87-40cf-93da-41946afb6615 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.424479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.425749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.515282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.516366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.517293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.517860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.518718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.520978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363c745-ad1c-4f50-9e00-1c1bdc160282 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.531526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 774/1545] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:00.536321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.550263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183af6b-1fa9-44d2-a9df-b7800ace504b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.560416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.561683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.562010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.562287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.562957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:00.563336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.564480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:00.564480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:17:00.564480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:17:00.571253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.573179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 772/1546] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:00.575250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.575804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.576362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.586190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 775/1547] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:00.587116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.606587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c0776-6b5c-453f-823e-0f7a8e435f3d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.609726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.611431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.626727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 773/1548] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.629176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.629176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.629405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.629531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.630282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.630884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.636398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e822b2e-e30f-457a-99f0-e954a15e08ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.659636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f1534-57d9-4784-aa5d-b3c513d89732 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.668864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.669821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.670485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.670979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.672034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.673408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:00.673408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:17:00.673408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:17:00.675346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.675501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 776/1549] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:00.677507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617785ad-f949-4989-8f42-7ad76e0e4be3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.677931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617785ad-f949-4989-8f42-7ad76e0e4be3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.678343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617785ad-f949-4989-8f42-7ad76e0e4be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.726662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 774/1550] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:00.727557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.728813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-617785ad-f949-4989-8f42-7ad76e0e4be3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.730024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-617785ad-f949-4989-8f42-7ad76e0e4be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.807772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 777/1551] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.809054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaca53-a8a8-4308-8b33-cf54ce920235 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.809342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaca53-a8a8-4308-8b33-cf54ce920235 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.809618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaca53-a8a8-4308-8b33-cf54ce920235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.814082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-e32b-4797-bfbf-31b79ce13ff2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.818813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.819585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.819873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.820180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.820832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.822469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1d0ec-6c96-463c-ae79-8e7949b37755 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.829792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6dfaca53-a8a8-4308-8b33-cf54ce920235 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.830986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6dfaca53-a8a8-4308-8b33-cf54ce920235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.836490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 775/1552] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:00.836490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a104-ffb8-4da4-a996-839adaef04c9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.836490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a104-ffb8-4da4-a996-839adaef04c9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.836490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a104-ffb8-4da4-a996-839adaef04c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.839121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 778/1553] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.840341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.840664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.840939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.860117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.860638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e6c4a104-ffb8-4da4-a996-839adaef04c9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.861254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.861920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6c4a104-ffb8-4da4-a996-839adaef04c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.869478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 779/1554] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.870046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.870612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 776/1555] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.871676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc014-62d2-40ae-b526-d125be73413a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.871992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc014-62d2-40ae-b526-d125be73413a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.872285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc014-62d2-40ae-b526-d125be73413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.889733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c33e9d-6f1b-4b3e-a210-c27ba57d709a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.894982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.895627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.895908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.896179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.896610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fcdfc014-62d2-40ae-b526-d125be73413a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.896813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.897842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:00.897842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:17:00.897842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:17:00.898094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fcdfc014-62d2-40ae-b526-d125be73413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.899054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.906612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 777/1556] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.907709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.908016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.908298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.924673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 780/1557] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/users => generated 302 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:00.928250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.930815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.932037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.935723 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90331a7d-8fb8-40d9-bbbf-c6d0a253fc4f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.947481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 778/1558] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.947686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.947794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.947874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034ff08-51aa-4e2f-94d7-f327c1621c63 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.948287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:00.953268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81129ea-b9c4-4ec6-8992-41f9ad5ff9d4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:00.958637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:00.959149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.959499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.959786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] PATH_INFO: `/v3/projects/bfb2171b0afc4bd3bf84b14b9649031f/users/3fa82f9d9a924d7092d48a4f55382061/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.961856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 781/1559] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:00.963108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffade-de37-469e-b2f7-94e5151e16e3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.963427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffade-de37-469e-b2f7-94e5151e16e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:00.963708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffade-de37-469e-b2f7-94e5151e16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:00.977058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:00.985346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a7bffade-de37-469e-b2f7-94e5151e16e3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:00.987908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a7bffade-de37-469e-b2f7-94e5151e16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:00.988167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb2171b0afc4bd3bf84b14b9649031f, user_id=3fa82f9d9a924d7092d48a4f55382061, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:00.988974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:00.988974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:17:00.988974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:17:00.991114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463ab2a-f33f-4965-8aa5-b97dbf250dd7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:00.996074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 782/1560] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:00.999320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cd946-52df-4233-8ab0-1c2805926617 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:00.999731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cd946-52df-4233-8ab0-1c2805926617 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.000093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cd946-52df-4233-8ab0-1c2805926617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.009375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 779/1561] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:00 2022] PUT /identity/v3/projects/bfb2171b0afc4bd3bf84b14b9649031f/users/3fa82f9d9a924d7092d48a4f55382061/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:01.015529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.015956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.016371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.028423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c84cd946-52df-4233-8ab0-1c2805926617 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:01.030450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c84cd946-52df-4233-8ab0-1c2805926617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:01.039744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:01.041203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:01.106314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 783/1562] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:01.108364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a426ac-d02f-42b4-ba87-7e07b00248d7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.108764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a426ac-d02f-42b4-ba87-7e07b00248d7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.109207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a426ac-d02f-42b4-ba87-7e07b00248d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.117394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 780/1563] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:01.119641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.133021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcef349-ac2f-4cce-a51d-2b011962dd51 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.134956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-91a426ac-d02f-42b4-ba87-7e07b00248d7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:01.136209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91a426ac-d02f-42b4-ba87-7e07b00248d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:01.139559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.140386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.140834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.141117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.141768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.143498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.145258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 784/1564] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:01.150251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.150544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.150820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.155250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 781/1565] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:01.155991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.168835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c22f-7ebb-4f53-88fa-126e9fccd076 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.172427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:01.173491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:01.174734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.175557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.175875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.176235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.176961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.178884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337a68e-97e2-45af-b004-c42d3fba612b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.182649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 785/1566] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:01.184052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.198410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff5d6-d490-4a86-aee4-d63647198a17 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.203248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.203868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.204362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.204653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.205478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:01.205800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.207748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.218138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 782/1567] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.230701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 786/1568] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:01.230701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.231006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.231006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.231006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.232087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3bf42-0ade-413c-92c4-a56d649a4b04 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.260327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] MFA Rules not processed for user `3fa82f9d9a924d7092d48a4f55382061`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:01.261902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:01.296655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.297484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.298754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.298754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.299056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:01.299486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.301258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.321280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 787/1569] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:01.331571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.337762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 783/1570] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:01.337762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.342070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd34a21-ea58-41c6-8a44-0d777fb23afc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.345423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee725-347c-4305-bc50-49646ae0b0d5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.347457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.348144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.348514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.348797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.349455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:01.349769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.350202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.350806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.351169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.351423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.351775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.352069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:01.352372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.353953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.365271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 788/1571] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:01.366141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.374479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 784/1572] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:01.374479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.378455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe874c-dd69-46eb-8a56-b2de7324954a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.379291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ad999-5f72-46b9-a235-776fb1b2adac admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.383549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.384176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.384458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.384599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.384767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.385093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.385438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:01.385551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.385766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.385952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.386539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.387458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.388216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.399636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 789/1573] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:01.400447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.400937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 785/1574] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:01.401585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.412697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17bda-665b-4e62-aa0a-7b649fb3c576 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.417098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c648c9-7c19-491e-9735-3c4ca78ed342 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.417550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.418194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.418501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.418800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.419487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.421543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.422258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.422850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.423134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.423470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.424112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.426017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.439848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:01.450916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 790/1575] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/users => generated 295 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.452485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.455080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 786/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.455993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.462754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca0b0d-ca53-45fe-93e8-936ca6cf3ca9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.475947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:01.523198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e88652-ebd6-43e9-bbec-03684338bd2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.525219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.525834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.526123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.527035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.527771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.528001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.528672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.529064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.529373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.530095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.530287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.532397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.567062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 791/1577] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 355 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.567751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 787/1578] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.568153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.568623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.683076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1e4cf-b761-442c-a26a-a96e5b80b7eb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.683568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47006e0-b740-449f-9147-76fd4048a577 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.683920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.683920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.683920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.683920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.684386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.684386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.684594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.684594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.684594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.684594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.685141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.685141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.694971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.695963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.703802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.707994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.760984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568c43-5c1c-46d8-95a6-c17e0d5cb961 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.770691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96522-2c59-4d5b-881c-b3cec49130d6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.775382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.775983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.776264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.776533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.777161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.779123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.790816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:01.792851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:01.808644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 789/1581] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.809637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.825500 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.826164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.826505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.827315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] PATH_INFO: `/v3/projects/87d136412e19448b9d1544395dc081fe/users/9f54103bca164ca18478762852379a78/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.841979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d136412e19448b9d1544395dc081fe, user_id=9f54103bca164ca18478762852379a78, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.844668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.857903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 793/1582] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:01 2022] PUT /identity/v3/projects/87d136412e19448b9d1544395dc081fe/users/9f54103bca164ca18478762852379a78/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:01.858780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.880814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a128-8fff-417f-9b3a-2655c6a4b5c0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.924637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad649a-af0a-4dd9-ad1d-b1bd9dfbb55e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.929440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.930050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.930330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.930603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.931250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.933302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.942034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.942655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.942942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.943250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.943904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.945601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.956890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 790/1584] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:01.957065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 794/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:01.960552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.960722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:01.971017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e9bd-fd0f-46ee-bb0d-990d42856c55 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.971916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e1d17-ce5f-44c7-8d7b-ed54dc291a61 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:01.975923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.976518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.976802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.977108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.977602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:01.977750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.978290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:01.978577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:01.978900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:01.979645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:01.979767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:01.981633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.002351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 795/1585] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:02.003174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.010955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 791/1586] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:01 2022] POST /identity/v3/users => generated 278 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:02.010955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.012832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f334b6-64eb-419c-a03a-a368c6b56ee1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.019462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.020066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.020344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.020617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.021249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.021664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2046127-0c1a-42ec-b5a8-a08e6a3dcb07 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.023424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.026590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.027197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.027504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.027781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.028417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.030396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52692-f406-45d6-8606-2fbff74d4e00 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.050295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 796/1587] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:02.050295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.055497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 792/1588] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:02.057812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.058026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.058483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eca1bd-3b11-4cd8-91eb-0049356ad3bd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.058614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.064010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.064689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.065016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.065295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.066013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.068175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.078905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:02.080219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:02.089597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 793/1589] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:02.090465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.094521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 797/1590] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:02.094521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.101975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9943e-deab-4e14-8e49-4bd44968546f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.107086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.107856 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.108267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.108544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.109031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff630e-6dc2-43e2-8204-1b2e84ba457c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.109221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.110815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.114386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.115153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.115582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.115905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.116955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.119328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.121331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 794/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.122583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.132496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 798/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.132911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b514d-f195-4d9b-924a-a755c0119bd0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.133643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.140131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.140731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.141013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.141287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.141930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.143549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.144997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a619e-82e0-407b-b129-e1748ab1efce admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.149757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.150494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.150768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.151047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.151742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.153333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.157071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 795/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.157071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.164696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d41cc-b79f-408e-a121-c8ab07e70baa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.171141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 799/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.171141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.171368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.171368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.171368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.171368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.171933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.173518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.179579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fdd59-4b6a-4399-8dc2-a0cb88c23190 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.184468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 796/1595] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.185262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.186701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.186971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.187271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.188127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.188361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.189909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.198606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cec64-2e83-4dac-87d6-2d98a4a96346 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.200855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 800/1596] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.201663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.206742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.207483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.207847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.208197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.208844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.210452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.224283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 797/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.224283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.225833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:02.231315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845825-8f3c-4e15-931c-709911b114d0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.237807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.238946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.238946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.239163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] PATH_INFO: `/v3/projects/d07b587d6ac148b0a7751c851260b3bc/users/c3dc3a92c6424cdbb3a2d3c1baaa95e3/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.242517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:02.255524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07b587d6ac148b0a7751c851260b3bc, user_id=c3dc3a92c6424cdbb3a2d3c1baaa95e3, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.258169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.266052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a235a-d8fe-487a-b755-c876117cb1b2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.272978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 798/1598] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/d07b587d6ac148b0a7751c851260b3bc/users/c3dc3a92c6424cdbb3a2d3c1baaa95e3/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.273679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.318371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.319035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.319378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.319675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.339605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e45501-da46-482c-aeb4-b7cf9391b759 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.345112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.345759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.346045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.346330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.346978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.348705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.359505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 799/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.360375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.369918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a014d5-7a7f-418b-b33c-fd00d5d9caa6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.375248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.377320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.377638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.377919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] PATH_INFO: `/v3/projects/f518bba39340457d835f9290d179b260/users/ee87c9b8759a4b19b49e813a11404d5f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.385558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.387015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.393844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f518bba39340457d835f9290d179b260, user_id=ee87c9b8759a4b19b49e813a11404d5f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.396664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.397491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 801/1600] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:02.398296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.408688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec24663-a17b-4f29-a286-4f7997c46ed4 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.413226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 800/1601] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/f518bba39340457d835f9290d179b260/users/ee87c9b8759a4b19b49e813a11404d5f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.413871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.476568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.477332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.477686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.478165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.478865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.480628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.483333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b66da-a8bc-4b69-ae86-fe84a37fe8e6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.489770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.490610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.490960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.491392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.492149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.494426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.495999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 802/1602] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.496905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.508726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 801/1603] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.509478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244e1f5-6c26-497f-83db-15b40bbe1c7a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.513677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.517509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.518357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.518784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.519109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.519887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.521535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514baada-bf96-4ca7-8e82-234eeb22632d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.525786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07375-90f0-4cff-b0cb-f53a6ceb3c10 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.531809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.532445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.532711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.532985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.533633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.535282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.537317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 803/1604] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.538862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.539284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.539667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.548278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 802/1605] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.549446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.559989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c5eca-478d-4f0d-87ad-6e44986e7819 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.565452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.566118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.566404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.566684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] PATH_INFO: `/v3/projects/8f871f9e81cc4408b579b65fc7537f3f/users/73e7e33675ac4547b43305072412c709/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.567588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:02.569173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:02.579093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 804/1606] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:02.579833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.587922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:02.589178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f871f9e81cc4408b579b65fc7537f3f, user_id=73e7e33675ac4547b43305072412c709, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.595439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.609763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a797a3c-8e97-4e41-9fd3-fa22def1bef1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.615562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 803/1607] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/8f871f9e81cc4408b579b65fc7537f3f/users/73e7e33675ac4547b43305072412c709/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.616387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.653752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:02.676598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.677269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.677552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.677830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.678532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.680234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.692150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 805/1608] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:02.697276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.704694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247819a1-87be-4057-bcc6-53f4b43f4e74 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.709298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bef0c0-dd37-4adb-91ae-962496d212aa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.711686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.712583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.712916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.713197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] PATH_INFO: `/v3/projects/fb75738585494f908021bd07549a2607/users/cdc7a21cc31145be8715c4a7e3c085a0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.716971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.718031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.718532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.718998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] PATH_INFO: `/v3/projects/6c148946e0fe4eadacb3cfcade050e5f/users/ca6d51cbceff489bae4222ff8888ac16/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.735513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb75738585494f908021bd07549a2607, user_id=cdc7a21cc31145be8715c4a7e3c085a0, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.738476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.741610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c148946e0fe4eadacb3cfcade050e5f, user_id=ca6d51cbceff489bae4222ff8888ac16, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.746184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.757450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 804/1609] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/fb75738585494f908021bd07549a2607/users/cdc7a21cc31145be8715c4a7e3c085a0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.758584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.772423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 806/1610] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/6c148946e0fe4eadacb3cfcade050e5f/users/ca6d51cbceff489bae4222ff8888ac16/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.776369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.844658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4bfdb-2643-4831-ab2d-080d8c117398 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.878574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bd35e-d99e-423c-8288-a3705053762e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:02.894302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.895138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.895588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.896440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] PATH_INFO: `/v3/projects/dff1326e1026492080d292aa3a157366/users/03eed7086183469f8cc1e765422bea87/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.917859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:02.918536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.918955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.919255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] PATH_INFO: `/v3/projects/1c0d98b877d6458cb0e4c49dc8a57485/users/ecf0e8a3fae04b499310fe0d35c08269/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:02.920902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff1326e1026492080d292aa3a157366, user_id=03eed7086183469f8cc1e765422bea87, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.925494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.937722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0d98b877d6458cb0e4c49dc8a57485, user_id=ecf0e8a3fae04b499310fe0d35c08269, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:02.941314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288759c9-0348-4125-919f-b043f4d75329 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:02.941467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 807/1611] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/dff1326e1026492080d292aa3a157366/users/03eed7086183469f8cc1e765422bea87/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.944437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:02.947109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:02.961755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 805/1612] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:02 2022] PUT /identity/v3/projects/1c0d98b877d6458cb0e4c49dc8a57485/users/ecf0e8a3fae04b499310fe0d35c08269/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:02.963389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:02.963782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:02.964131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.003311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] MFA Rules not processed for user `73e7e33675ac4547b43305072412c709`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.004613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.025049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681fef0-dbc2-4ac8-b366-51771a0aea86 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.025296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:03.039836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:03.083111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.083849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.084128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.084431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.084608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 806/1613] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.085268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.161020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0d2a-5289-4a4e-852d-6f32422fb8d3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.164255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.164929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.165257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.165614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] PATH_INFO: `/v3/projects/f518bba39340457d835f9290d179b260/users/ee87c9b8759a4b19b49e813a11404d5f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.167717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f518bba39340457d835f9290d179b260, user_id=ee87c9b8759a4b19b49e813a11404d5f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.170859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02ce2-6d8d-4b76-a916-fba364a814fc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.177140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.178729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdaee8d-81c4-4b59-9a5c-682d5f2a8e51 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.183972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 807/1614] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:03 2022] PUT /identity/v3/projects/f518bba39340457d835f9290d179b260/users/ee87c9b8759a4b19b49e813a11404d5f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:03.185125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b77ff-00ad-4d0c-ac14-91e336225964 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.185383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b77ff-00ad-4d0c-ac14-91e336225964 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.185666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b77ff-00ad-4d0c-ac14-91e336225964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.195703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 808/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:03.197125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a8455-78c1-41fb-add6-1a28553ff460 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.197522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a8455-78c1-41fb-add6-1a28553ff460 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.197861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a8455-78c1-41fb-add6-1a28553ff460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.216144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0b4b77ff-00ad-4d0c-ac14-91e336225964 None None] MFA Rules not processed for user `cdc7a21cc31145be8715c4a7e3c085a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.217278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b4b77ff-00ad-4d0c-ac14-91e336225964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.219388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c55a8455-78c1-41fb-add6-1a28553ff460 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.220634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c55a8455-78c1-41fb-add6-1a28553ff460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.294388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 809/1616] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.296924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a8286-0daf-40fb-a5fe-c10358d29317 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.297360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a8286-0daf-40fb-a5fe-c10358d29317 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.297786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a8286-0daf-40fb-a5fe-c10358d29317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.306735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 808/1617] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.308417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.309262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.311278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.324903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0c7a8286-0daf-40fb-a5fe-c10358d29317 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.326136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0c7a8286-0daf-40fb-a5fe-c10358d29317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.342716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.346705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.353305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 810/1618] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.353305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a238-8e89-4a38-996b-e0997af7647e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.353305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a238-8e89-4a38-996b-e0997af7647e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.353305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02a238-8e89-4a38-996b-e0997af7647e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.357205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 811/1619] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 30 16:17:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:03.359253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbfd16-4da6-4db8-96dc-71a1765930c1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.359805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbfd16-4da6-4db8-96dc-71a1765930c1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.360290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbfd16-4da6-4db8-96dc-71a1765930c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.362098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 809/1620] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.363117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.411122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-06fbfd16-4da6-4db8-96dc-71a1765930c1 None None] MFA Rules not processed for user `ee87c9b8759a4b19b49e813a11404d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.412236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06fbfd16-4da6-4db8-96dc-71a1765930c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.428873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:03.447708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd75b9-b93c-45e2-8fad-38d6c019c3bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.453148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.453996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.454538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.454844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.467913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.470139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.485352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 810/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:03.486315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.502680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a88906-26c5-4a52-9573-b785922ff5a8 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.509322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.509998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.510373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.510808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.511778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.514598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.524018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:03.563122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 811/1622] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/projects => generated 355 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:03.563984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.564447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 812/1623] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.565862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.566162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.566493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.639648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.641002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.671705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67569b43-8790-4c0a-9469-930ed5ac60b8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.676626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.677931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.678322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.678708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.742980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 813/1624] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.744994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.761837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef221d68-8d80-438f-bf86-c3fceb458257 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.767789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.768423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.768707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.768979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.783788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.785504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ad0e-34e7-482d-bf46-e558fd6650b4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.793722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.795278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.800318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 812/1625] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:03.801974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.802636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.803056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.807622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 814/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:03.808696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.820097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c156f-70e2-4a26-a64b-82aa64d65fa7 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.827147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.827872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.828693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.828811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.829709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:03.832212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:03.838483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] MFA Rules not processed for user `9f54103bca164ca18478762852379a78`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:03.839789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:03.862258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 815/1627] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/users => generated 284 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:03.863265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.918833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:03.925093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 813/1628] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:03.932463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:03.946042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b417c-8af5-4269-bd40-ca80255ef84c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.952478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84cbb-1f05-45d2-ad18-ff5311c591a5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:03.953770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.954394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.954680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.954957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:03.958416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:03.959083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:03.959406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:03.959891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.021603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.023262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.035836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 814/1629] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:04.036426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.041938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:04.047649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9a3f-c0e0-438f-9015-861ce2b7db31 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.054603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.055367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.055627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.056024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.057070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.059121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.065871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.067422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.076543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 815/1630] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:04.076543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.081044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 816/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:04.082047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.094793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a9df-24ef-4d2c-843f-fbc214ef4c3c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.101709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.102567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.102959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.103440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] PATH_INFO: `/v3/projects/97ff2cf14a6b407b8825ae28b9985b64/users/3c8efff2dd9a4899bdcbdd5292856d63/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.122297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1accb-955e-4d36-92d6-809267f3003a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.130810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.133399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.133399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.133399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.133399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.134065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ff2cf14a6b407b8825ae28b9985b64, user_id=3c8efff2dd9a4899bdcbdd5292856d63, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.136195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.147863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.176457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 817/1632] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:04 2022] PUT /identity/v3/projects/97ff2cf14a6b407b8825ae28b9985b64/users/3c8efff2dd9a4899bdcbdd5292856d63/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:04.178420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.218320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 816/1633] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:04 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:04.219835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.355096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49923c11-c66b-447b-b524-ea6bae319dcb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.360447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.361362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.361654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.362059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.363374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.365656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.369229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afee3-35c5-4bca-8140-af231d14aa91 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.392031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.392657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.392934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.393274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.393936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.397853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.414237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 817/1634] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:04 2022] POST /identity/v3/users => generated 308 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:04.415480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.430430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 818/1635] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:04.431448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.449872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0a8dd-f15b-4d1d-8f04-7313793f0f48 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.462913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.463510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.463788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.464131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.464927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.466602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2f0be-0c8e-4ad7-aec3-24fc9382ebe3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.492788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 819/1636] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:04.498438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc592-134c-4a90-9c50-5b5a7d74f8df None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.498709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc592-134c-4a90-9c50-5b5a7d74f8df None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.498987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc592-134c-4a90-9c50-5b5a7d74f8df None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.503039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 820/1637] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 30 16:17:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:04.503888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc592-134c-4a90-9c50-5b5a7d74f8df None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.533966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc592-134c-4a90-9c50-5b5a7d74f8df None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.536102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78ea8c-4c37-4b75-9f6b-ccbb0754dc79 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.540472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.541294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.541617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.541886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.543713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.544513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.545134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.545610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] PATH_INFO: `/v3/projects/97ff2cf14a6b407b8825ae28b9985b64/users/3c8efff2dd9a4899bdcbdd5292856d63/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.548700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ff2cf14a6b407b8825ae28b9985b64, user_id=3c8efff2dd9a4899bdcbdd5292856d63, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.553040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.559837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:04.570980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 821/1638] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:04 2022] PUT /identity/v3/projects/97ff2cf14a6b407b8825ae28b9985b64/users/3c8efff2dd9a4899bdcbdd5292856d63/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:04.573812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:04.682077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718025c-95e1-4d8d-8ce6-7c70ed37240e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:04.682463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.683841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b14e25-91de-4c3b-8a52-6798ad36e8ad service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.690946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:04.692300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.692814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.693344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] PATH_INFO: `/v3/projects/2602f691c6a44fecb0b72ef124143d0a/users/a35d1c2e35c64c35a931cf71619e4117/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.711268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 818/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:04.717617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2602f691c6a44fecb0b72ef124143d0a, user_id=a35d1c2e35c64c35a931cf71619e4117, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:04.718534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.718653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.718966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.722143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8917d0-78c5-45bd-870d-1d440899c6ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:04.748600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 822/1640] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:04 2022] PUT /identity/v3/projects/2602f691c6a44fecb0b72ef124143d0a/users/a35d1c2e35c64c35a931cf71619e4117/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:04.751417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:04.751682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:04.752252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:04.764823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:04.766078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:04.805805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:04.817614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] MFA Rules not processed for user `3c8efff2dd9a4899bdcbdd5292856d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:04.820696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:04.907609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 819/1641] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:04.909243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:05.001049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df953197-5b8a-45f2-a47c-3c683a73f153 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:05.006767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:05.007552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:05.007842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:05.008120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:05.009100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:05.013688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:05.033125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 823/1642] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:05.034344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:05.042896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 820/1643] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:05.044932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:05.064315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146783a-e3f9-42c4-92b1-88c9ddb39d5a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:05.079095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:05.080183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:05.081637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:05.082653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] PATH_INFO: `/v3/projects/2602f691c6a44fecb0b72ef124143d0a/users/a35d1c2e35c64c35a931cf71619e4117/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:05.087946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2602f691c6a44fecb0b72ef124143d0a, user_id=a35d1c2e35c64c35a931cf71619e4117, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:05.098560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:05.128995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 821/1644] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:05 2022] PUT /identity/v3/projects/2602f691c6a44fecb0b72ef124143d0a/users/a35d1c2e35c64c35a931cf71619e4117/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:05.160523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ccc4c-6b01-4a57-b8d1-dc5fc4529c1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:05.231917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:05.250022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:05.251340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:05.251877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:05.252379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:05.296720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:05.366724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffe5b3-20ab-4c9f-b92a-9151bb207fad admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:05.372734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:05.376025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:05.376576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:05.377182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:05.379253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:05.386011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:05.389434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:05.416259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 824/1645] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:05.471644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:05.473600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e241932-8743-491c-8b9f-779352c4182b service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:05.488492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 822/1646] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:06.325173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:06.335283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d008f2-92f9-449f-a91b-8771153b6a31 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:06.347866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:06.348755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:06.348874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:06.351893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:06.435359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:06.437031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d03dbe-7f92-4bdb-9bad-bd575091f484 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:06.449201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 825/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:06.936932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46678-1854-4686-b4ad-821ac0dbe7a6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:06.937435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46678-1854-4686-b4ad-821ac0dbe7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:06.937435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46678-1854-4686-b4ad-821ac0dbe7a6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:06.938831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 823/1648] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 30 16:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:06.957428 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a9bb-6bd1-4841-9823-9f6dec804caf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:06.958026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a9bb-6bd1-4841-9823-9f6dec804caf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:06.958544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a9bb-6bd1-4841-9823-9f6dec804caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:06.994480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3192a9bb-6bd1-4841-9823-9f6dec804caf None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:06.996165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3192a9bb-6bd1-4841-9823-9f6dec804caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:07.011260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 826/1649] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 30 16:17:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:07.012406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950ff91-6bd7-454b-82df-bf50fbfc21d8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:07.012697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950ff91-6bd7-454b-82df-bf50fbfc21d8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:07.012976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950ff91-6bd7-454b-82df-bf50fbfc21d8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:07.014883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 824/1650] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 30 16:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 16:17:07.021536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e272e-c8b9-4e77-bb04-68265f0190d9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:07.022151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e272e-c8b9-4e77-bb04-68265f0190d9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:07.022787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e272e-c8b9-4e77-bb04-68265f0190d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:07.044311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-044e272e-c8b9-4e77-bb04-68265f0190d9 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:07.046679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-044e272e-c8b9-4e77-bb04-68265f0190d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:07.063543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 827/1651] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:17:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:07.072283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950ff91-6bd7-454b-82df-bf50fbfc21d8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:07.088625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950ff91-6bd7-454b-82df-bf50fbfc21d8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:07.098198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:07.100269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:07.101327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:07.101884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:07.114832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:07.117572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43180e3a-76fd-45d1-81e2-7073724b2fa7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:07.154358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 825/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:09.521252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f935b-3302-4eb3-848f-7ae9194e67ac None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:09.521798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f935b-3302-4eb3-848f-7ae9194e67ac None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:09.521798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f935b-3302-4eb3-848f-7ae9194e67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:09.640773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-170f935b-3302-4eb3-848f-7ae9194e67ac None None] MFA Rules not processed for user `a35d1c2e35c64c35a931cf71619e4117`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:09.642233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-170f935b-3302-4eb3-848f-7ae9194e67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:09.868912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 828/1653] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:12.575038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1956c-6e82-45e6-bc70-231fad82a36d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:12.576248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1956c-6e82-45e6-bc70-231fad82a36d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:12.576734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1956c-6e82-45e6-bc70-231fad82a36d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:12.578703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 826/1654] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:12.606875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9340-70cf-417b-b486-bd1fcd999e60 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:12.607433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9340-70cf-417b-b486-bd1fcd999e60 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:12.608318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9340-70cf-417b-b486-bd1fcd999e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:12.644272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-91aa9340-70cf-417b-b486-bd1fcd999e60 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:12.645469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91aa9340-70cf-417b-b486-bd1fcd999e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:12.771455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 829/1655] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:17:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:12.807043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1956c-6e82-45e6-bc70-231fad82a36d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:12.829555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1956c-6e82-45e6-bc70-231fad82a36d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:12.840643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:12.841358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:12.841640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:12.841966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:12.860910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:12.864753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12223a-65ad-48bf-a519-fe7924b1ff73 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:12.884640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 827/1656] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:14.862432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422d7c-4914-44ff-9203-7c7e11fed60f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:14.863098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422d7c-4914-44ff-9203-7c7e11fed60f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:14.863188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422d7c-4914-44ff-9203-7c7e11fed60f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:14.864945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 830/1657] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:14.974339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287725-26bd-4693-9689-f108ce36f4f5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:14.975446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287725-26bd-4693-9689-f108ce36f4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:14.975897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287725-26bd-4693-9689-f108ce36f4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.020152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e3287725-26bd-4693-9689-f108ce36f4f5 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:15.022589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e3287725-26bd-4693-9689-f108ce36f4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:15.049594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 828/1658] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:17:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:15.061975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422d7c-4914-44ff-9203-7c7e11fed60f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:15.067754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:15.150399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489924-98ca-4109-a51d-9e68531bd357 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:15.150868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489924-98ca-4109-a51d-9e68531bd357 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:15.151419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489924-98ca-4109-a51d-9e68531bd357 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.153265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 829/1659] 10.222.0.57 () {54 vars in 961 bytes} [Tue Aug 30 16:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:15.160053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:15.160483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:15.160914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.211474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] MFA Rules not processed for user `4c81e08e950648be89483f26f546a7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:15.212646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:15.232878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422d7c-4914-44ff-9203-7c7e11fed60f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:15.236556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 830/1660] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Aug 30 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:15.245918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:15.254291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:15.254938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:15.255266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:15.255514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] PATH_INFO: `/v3/users/9f54103bca164ca18478762852379a78` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.259679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f54103bca164ca18478762852379a78)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:15.261752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:15.274169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0c5ec-6207-4e68-9cd7-5fa67a440196 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:15.293815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:15.294473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:15.294812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:15.295669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.311657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:15.313385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19e2eb-4ed4-45e2-8406-795b02530b27 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:15.338191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 831/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:15.364395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f54103bca164ca18478762852379a78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:15.381237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f54103bca164ca18478762852379a78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:15.384731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f54103bca164ca18478762852379a78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:15.402261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f54103bca164ca18478762852379a78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:15.428674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f54103bca164ca18478762852379a78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:15.444013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 831/1662] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:15 2022] DELETE /identity/v3/users/9f54103bca164ca18478762852379a78 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:15.764105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:15.764245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:15.767968 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:15.842825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] MFA Rules not processed for user `c3dc3a92c6424cdbb3a2d3c1baaa95e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:15.844914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:16.136658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 832/1663] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:16.161633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:16.163196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:16.370705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba84b-5cdb-4304-8385-95bd99b094d0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:16.391192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:16.391857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:16.392144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:16.392424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:16.425567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:16.427377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:16.454538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 832/1664] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:16.802308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:16.805140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:16.847952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:17.111264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38475a05-c1be-44ac-8a56-8fccf6afb79c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:17.129314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:17.130078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:17.130396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:17.130675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] PATH_INFO: `/v3/projects/87d136412e19448b9d1544395dc081fe` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:17.137717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d136412e19448b9d1544395dc081fe)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:17.141852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:17.254527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d136412e19448b9d1544395dc081fe', 'request_id': 'req-a617ad63-94a8-4e22-8917-e0436a7c63c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.273858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d136412e19448b9d1544395dc081fe', 'request_id': 'req-a617ad63-94a8-4e22-8917-e0436a7c63c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.300804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d136412e19448b9d1544395dc081fe', 'request_id': 'req-a617ad63-94a8-4e22-8917-e0436a7c63c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.336773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] The token cache is being invalidate because project 87d136412e19448b9d1544395dc081fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:17.336773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.340717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a617ad63-94a8-4e22-8917-e0436a7c63c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.346312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 833/1665] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:16 2022] DELETE /identity/v3/projects/87d136412e19448b9d1544395dc081fe => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:17.349989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:17.369757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d45d1-69ea-49f4-8515-b9092c53ff82 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:17.382531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:17.506093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:17.507093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:17.507724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:17.508317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] PATH_INFO: `/v3/users/a35d1c2e35c64c35a931cf71619e4117` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:17.511389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35d1c2e35c64c35a931cf71619e4117)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:17.515530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:17.678879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35d1c2e35c64c35a931cf71619e4117'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.705218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35d1c2e35c64c35a931cf71619e4117'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.707664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35d1c2e35c64c35a931cf71619e4117'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.735778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35d1c2e35c64c35a931cf71619e4117'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.777025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35d1c2e35c64c35a931cf71619e4117'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:17.798392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 833/1666] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:17 2022] DELETE /identity/v3/users/a35d1c2e35c64c35a931cf71619e4117 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:18.576359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9bf5d-56fe-46f8-a96b-b908f2d7ee1d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:18.577185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9bf5d-56fe-46f8-a96b-b908f2d7ee1d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:18.577614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9bf5d-56fe-46f8-a96b-b908f2d7ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:18.661812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-41b9bf5d-56fe-46f8-a96b-b908f2d7ee1d None None] MFA Rules not processed for user `ca6d51cbceff489bae4222ff8888ac16`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:18.663834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-41b9bf5d-56fe-46f8-a96b-b908f2d7ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:18.828721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:18.829983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:18.859637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 834/1667] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:18.869890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:18.869890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:18.869890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:18.940264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] MFA Rules not processed for user `ca6d51cbceff489bae4222ff8888ac16`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:18.941981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:18.956984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 835/1668] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:18.968508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:18.970183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:19.002163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d3a7c-22f1-4a59-acc0-c6513452b105 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.023499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.024165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.024493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.024765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] PATH_INFO: `/v3/projects/2602f691c6a44fecb0b72ef124143d0a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.031988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2602f691c6a44fecb0b72ef124143d0a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.039684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:19.039684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Aug 30 16:17:19.039684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: Aug 30 16:17:19.043130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.063562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:19.152678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:19.220031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2602f691c6a44fecb0b72ef124143d0a', 'request_id': 'req-b2d25a85-5988-44c5-a686-43f5a67a4164'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:19.240499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2602f691c6a44fecb0b72ef124143d0a', 'request_id': 'req-b2d25a85-5988-44c5-a686-43f5a67a4164'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:19.250213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294e31-701e-4108-ad73-9fbdb7a2ed2c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.266649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2602f691c6a44fecb0b72ef124143d0a', 'request_id': 'req-b2d25a85-5988-44c5-a686-43f5a67a4164'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:19.283095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] The token cache is being invalidate because project 2602f691c6a44fecb0b72ef124143d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:19.284261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:19.284597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b2d25a85-5988-44c5-a686-43f5a67a4164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:19.290374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 834/1669] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:18 2022] DELETE /identity/v3/projects/2602f691c6a44fecb0b72ef124143d0a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:19.291751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405636d7-01a4-42d7-acdf-7fd1387501f9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.292047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405636d7-01a4-42d7-acdf-7fd1387501f9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.292331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405636d7-01a4-42d7-acdf-7fd1387501f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.321369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-405636d7-01a4-42d7-acdf-7fd1387501f9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:19.322699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-405636d7-01a4-42d7-acdf-7fd1387501f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:19.428709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 835/1670] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:19.430504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.430719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.431039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.431484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.432153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.432433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.432707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.435177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:19.458681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:19.462111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:19.472221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 836/1671] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:19.478576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:19.492742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ff9b-a766-47bc-9683-4a8dfd03b1b2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.503510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.504213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.504496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.504917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.505596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:19.505951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.515941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.543988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 837/1672] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:19.561009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:19.577061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ebfaf-46ac-404c-94ee-6285e6096d9f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.610259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.610891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.611188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.611606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.613021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.615271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.633457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.635325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.668474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 836/1673] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:19.688471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 838/1674] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:19 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:19.689316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:19.707315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f46c-4732-4a1e-84d1-d12ba8a5df0c service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.840367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:19.855397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.856038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.858003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.859385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.860130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.865283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.916922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 837/1675] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:19 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:19.922726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:19.939955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aff55-d4ab-4df3-80f4-4feabe8e7b5d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:19.963107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:19.963814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:19.964100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:19.964375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:19.965050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:19.967158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:19.997089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:20.000778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 839/1676] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:20.014895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49575e-336b-4906-b1db-917a6fe2734e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:20.033187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:20.033892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:20.034177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:20.034507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:20.038333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:20.040239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd166f1-838b-4540-bd42-c6619b3663f6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:20.105039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 838/1677] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:20.108318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:20.127091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814203-900a-4b3c-a353-87806328824c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:20.138714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:20.140386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:20.140927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:20.141420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] PATH_INFO: `/v3/projects/e594f292f8a148609ecbe4f1ea96c570/users/72e7c43452de4c50a2b133ba54c2060b/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:20.166037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e594f292f8a148609ecbe4f1ea96c570, user_id=72e7c43452de4c50a2b133ba54c2060b, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:20.169566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:20.202106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 840/1678] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:20 2022] PUT /identity/v3/projects/e594f292f8a148609ecbe4f1ea96c570/users/72e7c43452de4c50a2b133ba54c2060b/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:20.202864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:20.203160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:20.203460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:20.283774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] MFA Rules not processed for user `72e7c43452de4c50a2b133ba54c2060b`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:20.285151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:20.507606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 839/1679] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:20.512022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:20.650089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:20.687207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d57f6-045e-4a19-b242-3ea471120a68 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:20.693792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:20.695121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:20.695121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:20.695409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:20.696196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:20.698449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:20.744216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:20.744720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 841/1680] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:20 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:20.774376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67410cfc-508d-4726-821d-47741ca46a9b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:20.939813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:20.940470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:20.940751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:20.941026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:20.941686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:20.943672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:20.994192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 840/1681] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:20 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:21.000007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.017360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab6ff7-f129-45fc-b657-3a0e257f165b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:21.035034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:21.035732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.036024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.036296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.037019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:21.038673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2f4a8-f279-40de-8035-a32c14e8cd4c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:21.071358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.071705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 842/1682] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:21.096683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e2a1d-1c58-47cc-9baa-1e963bc55945 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:21.125363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:21.127615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.128077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.128356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] PATH_INFO: `/v3/projects/a615235005714840bbbaafac369ddc30/users/7cf3c1e5c3194527b1a637ae3007993f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.187596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a615235005714840bbbaafac369ddc30, user_id=7cf3c1e5c3194527b1a637ae3007993f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:21.191521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b2a4-372a-457c-b126-f368ebe54c91 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:21.219814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dbb2-2f42-4b5b-8da7-f5274746ff53 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.220071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dbb2-2f42-4b5b-8da7-f5274746ff53 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.220403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dbb2-2f42-4b5b-8da7-f5274746ff53 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.221687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 843/1683] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 30 16:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:21.236080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 841/1684] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:21 2022] PUT /identity/v3/projects/a615235005714840bbbaafac369ddc30/users/7cf3c1e5c3194527b1a637ae3007993f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:21.236909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0dbb2-2f42-4b5b-8da7-f5274746ff53 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.245464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.249154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.251476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.349751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0dbb2-2f42-4b5b-8da7-f5274746ff53 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:21.358959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:21.360059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.361414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.361714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.362633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:21.365235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:21.382323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 844/1685] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:21.384074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.408466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:21.409951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:21.413569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732162e-9063-40d3-a4c1-ef041e2494c3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:21.426558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:21.427288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.427543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.428116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] PATH_INFO: `/v3/projects/a615235005714840bbbaafac369ddc30/users/7cf3c1e5c3194527b1a637ae3007993f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.432780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a615235005714840bbbaafac369ddc30, user_id=7cf3c1e5c3194527b1a637ae3007993f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:21.436009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5b1fa-ac79-45ca-af6c-973e9ffdf52a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:21.474596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:21.493042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 845/1686] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:21 2022] PUT /identity/v3/projects/a615235005714840bbbaafac369ddc30/users/7cf3c1e5c3194527b1a637ae3007993f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:21.505383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.505826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.506352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.582599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 842/1687] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:21.598920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] MFA Rules not processed for user `7cf3c1e5c3194527b1a637ae3007993f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:21.603713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:21.652366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.653859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:21.677878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46956d55-1d39-43d0-9686-f4a0686bde3d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:21.852063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:21.852774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:21.853009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:21.853301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:21.876310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:21.877800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:21.914108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 846/1688] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:21.915491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 843/1689] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:21.991816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:21.994501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:22.098662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:22.251484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862fc60-57cd-4a30-8edd-50553e7fb998 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:22.267306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:22.271364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:22.272076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:22.273679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:22.298997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:22.300495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:22.345300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 847/1690] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:22.506748 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:22.509204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:22.518607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:22.519902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:22.539981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67cbf-b696-4722-a3f6-e57920b3bbaa service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:22.559475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:22.560283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:22.560503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:22.560767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:22.689753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1003c-da59-46f4-bee5-2e70395e9bd4 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:22.696042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:22.701811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:22.702486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:22.702970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] PATH_INFO: `/v3/users/3fa82f9d9a924d7092d48a4f55382061` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:22.704672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa82f9d9a924d7092d48a4f55382061)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:22.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 16:17:22.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 30 16:17:22.706009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: Aug 30 16:17:22.707808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:22.738646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:22.739809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:22.763123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 848/1691] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:22.764068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:22.772115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:22.801340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc442-0d4d-4a3b-ae6b-33848c0162fe service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:22.815585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:22.817392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:22.817680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:22.818009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] PATH_INFO: `/v3/users/73e7e33675ac4547b43305072412c709` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:22.819984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e7e33675ac4547b43305072412c709)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:22.825411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:22.911744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa82f9d9a924d7092d48a4f55382061'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.945681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e7e33675ac4547b43305072412c709'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.952073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa82f9d9a924d7092d48a4f55382061'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.964310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e7e33675ac4547b43305072412c709'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.966059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa82f9d9a924d7092d48a4f55382061'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.966059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa82f9d9a924d7092d48a4f55382061'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.966832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e7e33675ac4547b43305072412c709'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.979326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e7e33675ac4547b43305072412c709'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.982953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2004f2a-81e6-4fcb-8fff-5ab73286cead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa82f9d9a924d7092d48a4f55382061'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:22.998775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f3864ba-b9b9-4d17-b723-f57c4a36c652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e7e33675ac4547b43305072412c709'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:23.013565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 849/1692] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:22 2022] DELETE /identity/v3/users/73e7e33675ac4547b43305072412c709 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:23.017746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49548cf-4e1c-4926-96df-f24097b0576f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.017746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49548cf-4e1c-4926-96df-f24097b0576f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.017746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49548cf-4e1c-4926-96df-f24097b0576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.027187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:23.063389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 844/1693] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:22 2022] DELETE /identity/v3/users/3fa82f9d9a924d7092d48a4f55382061 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:23.064953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898ac7-431c-47bf-a3c6-1d450d701e17 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.065262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898ac7-431c-47bf-a3c6-1d450d701e17 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.065536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898ac7-431c-47bf-a3c6-1d450d701e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.083663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b49548cf-4e1c-4926-96df-f24097b0576f None None] MFA Rules not processed for user `03eed7086183469f8cc1e765422bea87`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.085201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b49548cf-4e1c-4926-96df-f24097b0576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.117596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b7898ac7-431c-47bf-a3c6-1d450d701e17 None None] MFA Rules not processed for user `c3dc3a92c6424cdbb3a2d3c1baaa95e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.121678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b7898ac7-431c-47bf-a3c6-1d450d701e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.144746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:23.259851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 850/1694] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.261268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2c1e-45ff-4f57-be56-126c122e2d96 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.262588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2c1e-45ff-4f57-be56-126c122e2d96 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.262901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2c1e-45ff-4f57-be56-126c122e2d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.286312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 845/1695] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.287940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b93cd-3627-4f7a-9bb0-223f76d81cd9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.288419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b93cd-3627-4f7a-9bb0-223f76d81cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.288811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b93cd-3627-4f7a-9bb0-223f76d81cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.311801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-133e2c1e-45ff-4f57-be56-126c122e2d96 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.319633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-133e2c1e-45ff-4f57-be56-126c122e2d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.328505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-308b93cd-3627-4f7a-9bb0-223f76d81cd9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.329951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-308b93cd-3627-4f7a-9bb0-223f76d81cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.466981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 846/1696] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.468983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.469577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.471414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.498171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 851/1697] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.502724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.503958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.504314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.544158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] MFA Rules not processed for user `ecf0e8a3fae04b499310fe0d35c08269`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.548918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.564765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] MFA Rules not processed for user `03eed7086183469f8cc1e765422bea87`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:23.570560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.585408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 852/1698] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.585408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:23.585408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.724887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 847/1699] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:23.724887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:23.724887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.734917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806192c-7561-4a93-9b45-1e08d25c0983 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:23.754896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:23.755586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.756232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.756527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.759831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.784800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:23.787674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:23.789105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:23.803897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 853/1700] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:23.804972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:23.807393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.854268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7228f-6201-40a2-acfd-903d80b9f39a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:23.860021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:23.866135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.866488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.866774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.878509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:23.881007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:23.919312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 848/1701] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:23.934453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:23.936460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:23.939378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:23.946211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6206603-347c-453e-bf9c-cf800176c101 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:23.958900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:23.959820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:23.960197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:23.961409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:23.984043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:23.987788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:24.015068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 854/1702] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:24.198990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce69b9-3782-4680-a9ca-349e4a55302e service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:24.224326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:24.227091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:24.241131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:24.241131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:24.273615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:24.280573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f48618-6076-4fa1-96c1-3c4ae3d70131 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:24.334053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 849/1703] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:24.590319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:24.593360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:24.636146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713e96c-1d54-44a6-ae77-82858c5224e7 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:24.660138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:24.673418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:24.678122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:24.679563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:24.718005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:24.723837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b588-d41d-4c62-88e0-b1ed91f77418 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:24.754608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 855/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:25.802653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d935a-1346-4cc6-a795-871a25056464 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:25.803813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d935a-1346-4cc6-a795-871a25056464 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:25.804119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d935a-1346-4cc6-a795-871a25056464 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:25.805708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 850/1705] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 30 16:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:25.811545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:25.812409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:25.812868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:25.846652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:25.850061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:25.871791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 856/1706] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:25.917854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d935a-1346-4cc6-a795-871a25056464 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:25.955759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d935a-1346-4cc6-a795-871a25056464 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:25.962653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:25.963958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:25.964675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:25.965111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:25.990740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:26.000063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:26.014844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:26.016657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:26.049605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 851/1707] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:26.052471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1174f-2165-4ec6-9931-e4d5a3b68a1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:26.073058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:26.074170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:26.075782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:26.076966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] PATH_INFO: `/v3/projects/bfb2171b0afc4bd3bf84b14b9649031f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:26.080542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb2171b0afc4bd3bf84b14b9649031f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:26.095099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:26.182106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:26.190669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:26.229086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f235a-f91e-4647-881e-8e5adfee6ae9 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:26.247112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:26.249002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:26.249932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:26.252638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] PATH_INFO: `/v3/projects/8f871f9e81cc4408b579b65fc7537f3f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:26.255511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f871f9e81cc4408b579b65fc7537f3f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:26.258628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:26.300656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb2171b0afc4bd3bf84b14b9649031f', 'request_id': 'req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.328209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb2171b0afc4bd3bf84b14b9649031f', 'request_id': 'req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.362910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb2171b0afc4bd3bf84b14b9649031f', 'request_id': 'req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.420212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] The token cache is being invalidate because project bfb2171b0afc4bd3bf84b14b9649031f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:26.429217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.442299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.454694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 857/1708] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:26 2022] DELETE /identity/v3/projects/bfb2171b0afc4bd3bf84b14b9649031f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:26.460924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:26.462751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:26.508675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f871f9e81cc4408b579b65fc7537f3f', 'request_id': 'req-6c78b0bd-aec9-4992-af38-78d98a7482f2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.557609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f871f9e81cc4408b579b65fc7537f3f', 'request_id': 'req-6c78b0bd-aec9-4992-af38-78d98a7482f2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.587022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f871f9e81cc4408b579b65fc7537f3f', 'request_id': 'req-6c78b0bd-aec9-4992-af38-78d98a7482f2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.632588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] The token cache is being invalidate because project 8f871f9e81cc4408b579b65fc7537f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:26.635885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.637303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6c78b0bd-aec9-4992-af38-78d98a7482f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:26.648945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 852/1709] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:26 2022] DELETE /identity/v3/projects/8f871f9e81cc4408b579b65fc7537f3f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:26.655294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c64b-1947-480b-9343-4e4c22022259 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:26.659610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c64b-1947-480b-9343-4e4c22022259 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:26.659952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c64b-1947-480b-9343-4e4c22022259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:26.732991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bf4f6-d20e-4a7e-85b5-d6bbea8ede94 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:26.736614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e6c4c64b-1947-480b-9343-4e4c22022259 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:26.741395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6c4c64b-1947-480b-9343-4e4c22022259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:26.830794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:27.038312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:27.041766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:27.042893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:27.044440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] PATH_INFO: `/v3/users/ee87c9b8759a4b19b49e813a11404d5f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:27.047981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee87c9b8759a4b19b49e813a11404d5f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:27.052253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 853/1710] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:27.054240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:27.061403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:27.062898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e078-4d3a-4939-99be-65635b40302a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:27.065369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e078-4d3a-4939-99be-65635b40302a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:27.066698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e078-4d3a-4939-99be-65635b40302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:27.147453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9377e078-4d3a-4939-99be-65635b40302a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:27.152844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9377e078-4d3a-4939-99be-65635b40302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:27.181954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee87c9b8759a4b19b49e813a11404d5f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:27.214014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee87c9b8759a4b19b49e813a11404d5f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:27.215122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee87c9b8759a4b19b49e813a11404d5f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:27.215367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 854/1711] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:27.222519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377e078-4d3a-4939-99be-65635b40302a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:27.247050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee87c9b8759a4b19b49e813a11404d5f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:27.266373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377e078-4d3a-4939-99be-65635b40302a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:27.272177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee87c9b8759a4b19b49e813a11404d5f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:27.284016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:27.298242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:27.298242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:27.298242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:27.298242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:27.298242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:27.304022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 858/1712] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:26 2022] DELETE /identity/v3/users/ee87c9b8759a4b19b49e813a11404d5f => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:27.304756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:27.381651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:27.383081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:27.383276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 855/1713] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:27.670601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a632d2-6327-4d0a-9f8c-50fc1c8fcb08 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:27.686988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:27.687693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:27.688006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:27.688454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:27.689177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:27.691247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb72f3b-883c-4a9c-abb6-c5dd41e6c8b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:27.760732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 859/1714] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:27 2022] POST /identity/v3/projects => generated 375 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:27.765363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:27.790291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04792e-d344-4d09-b1cb-844c98a67577 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:27.909692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c0c69-c680-4a7e-80bd-4b12902200cf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:27.909937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c0c69-c680-4a7e-80bd-4b12902200cf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:27.910217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c0c69-c680-4a7e-80bd-4b12902200cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:27.937271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4e8c0c69-c680-4a7e-80bd-4b12902200cf None None] MFA Rules not processed for user `03eed7086183469f8cc1e765422bea87`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:27.938673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4e8c0c69-c680-4a7e-80bd-4b12902200cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:28.020752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:28.021409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.028468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.033289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.039750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:28.052607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:28.134871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 860/1715] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:28.136401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.136695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.136973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.148253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 856/1716] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:27 2022] POST /identity/v3/users => generated 288 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:28.158287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:28.185102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ae3e-aea6-42ac-b737-b82fa11647fd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:28.192494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:28.193377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.193800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.194225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.195169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:28.197584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:28.210735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 857/1717] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:28.215463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:28.217302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:28.241296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ef608-8a95-4873-bedc-ae68e7e4d4f8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:28.248800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:28.249883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.250303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.250861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.251716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:28.253592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:28.266458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 858/1718] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:28.272041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:28.298019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007fb3b-2d3a-4de2-b3d3-75e3bdd1e778 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:28.314697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:28.315466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.315691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.315967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] PATH_INFO: `/v3/projects/1437e428c0ef4f7daa655211a73c9664/users/453f0791592c413da9a2399a61c7ec38/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.353175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:28.354485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:28.355512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1437e428c0ef4f7daa655211a73c9664, user_id=453f0791592c413da9a2399a61c7ec38, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:28.359032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c06f8-3537-4132-82e4-1b3497fd3c9f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:28.393648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 859/1719] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:28 2022] PUT /identity/v3/projects/1437e428c0ef4f7daa655211a73c9664/users/453f0791592c413da9a2399a61c7ec38/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:28.405964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:28.406309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:28.406519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:28.497839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] MFA Rules not processed for user `453f0791592c413da9a2399a61c7ec38`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:28.499109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:28.522811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 861/1720] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:28.635704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:28.757015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 860/1721] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:28.773487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:28.777773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:28.793128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:28.795183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:28.825538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c8d33-55eb-48cd-8f4e-aa9d2787c534 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:29.016642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:29.019573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.020597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.021677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] PATH_INFO: `/v3/projects/f518bba39340457d835f9290d179b260` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.026457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f518bba39340457d835f9290d179b260)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:29.033073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:29.046530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cf19f-1a0e-46e6-ae55-3f908283c360 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:29.062393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:29.063544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.065734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.066103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.167683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f518bba39340457d835f9290d179b260', 'request_id': 'req-ef80e2db-3fde-446e-8430-ff3266b752c9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:29.170216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:29.171907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:29.180732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f518bba39340457d835f9290d179b260', 'request_id': 'req-ef80e2db-3fde-446e-8430-ff3266b752c9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:29.194715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f518bba39340457d835f9290d179b260', 'request_id': 'req-ef80e2db-3fde-446e-8430-ff3266b752c9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:29.196820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 861/1722] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:17:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:17:29.197761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:29.202105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:29.238166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] The token cache is being invalidate because project f518bba39340457d835f9290d179b260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:29.238476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:29.249297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ef80e2db-3fde-446e-8430-ff3266b752c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:29.266180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 862/1723] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:28 2022] DELETE /identity/v3/projects/f518bba39340457d835f9290d179b260 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:29.273344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dcf94-9eb9-49e6-9f6e-a17db7f76e00 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.273777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dcf94-9eb9-49e6-9f6e-a17db7f76e00 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.274480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dcf94-9eb9-49e6-9f6e-a17db7f76e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.357269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cb5dcf94-9eb9-49e6-9f6e-a17db7f76e00 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:29.361964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cb5dcf94-9eb9-49e6-9f6e-a17db7f76e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:29.398072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:29.459369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:29.538037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642405a8-90c6-47c0-a5e8-6bc9657ec018 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:29.608096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 863/1724] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:29.617476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.617930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.618398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.729024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:29.739605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:29.791852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 864/1725] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:29.793108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:29.820743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:29.822818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24396a73-9eed-4917-9785-f628170e8e6e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:29.825264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.825649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.826990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.830755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:29.839296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:29.840020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.840406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.840763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.841605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:29.842150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:29.847736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:29.878803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 865/1726] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:17:29.880065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:29.901050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006e84f-c3c5-4a73-b83d-ff8df10a9b13 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:29.916747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:29.917462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:29.917652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:29.917933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:29.918593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:29.921127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:29.939112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:29.941695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:29.961767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 862/1727] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:29.993457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 866/1728] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:29 2022] POST /identity/v3/projects => generated 409 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:29.999831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:30.025836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239f6f3-e8fd-402c-b71d-8a678e866fd3 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:30.210450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:30.211246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:30.211751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:30.212054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:30.212858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:30.214628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:30.286710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 863/1729] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:29 2022] POST /identity/v3/users => generated 305 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:30.291817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:30.335887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f319-f1c7-48a9-96b3-1c0e3df9d3e7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:30.356576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:30.357910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:30.358390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:30.358657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:30.359539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:30.370705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:30.419394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 867/1730] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:30.420553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:30.443769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445256f2-2af9-477e-823f-0bb5ea17d5a6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:30.453851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:30.454807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:30.455103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:30.455458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:30.456128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:30.458011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a76962-68d8-4460-9e20-52bc3100b451 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:30.487756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 864/1731] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:30.492114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:30.506094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314fd9-7804-4c1a-ba95-f3f27be8f0e2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:30.523136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:30.531602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:30.531810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:30.532154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] PATH_INFO: `/v3/projects/ad2d2a9a44bc4ac2a76848f27309fea4/users/795a62ad376b49c3bbffa3a302c764a2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:30.614986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2d2a9a44bc4ac2a76848f27309fea4, user_id=795a62ad376b49c3bbffa3a302c764a2, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:30.644207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:30.703245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 868/1732] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:30 2022] PUT /identity/v3/projects/ad2d2a9a44bc4ac2a76848f27309fea4/users/795a62ad376b49c3bbffa3a302c764a2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:30.715375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:30.715619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:30.715901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:30.775378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:30.804295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] MFA Rules not processed for user `795a62ad376b49c3bbffa3a302c764a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:30.805844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:30.998375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 865/1733] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:31.003373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.129742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.129921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf45cd-dd67-4cc3-a481-a5a7632c8313 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.131368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:31.149163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.150588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.151376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.152036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.153209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.157074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.160405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af097b-5762-4c9a-91f5-663bf35c543e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.172028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.173423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.173868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.174210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] PATH_INFO: `/v3/users/453f0791592c413da9a2399a61c7ec38` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.175917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=453f0791592c413da9a2399a61c7ec38)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.177932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.206747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 869/1734] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:31 2022] POST /identity/v3/projects => generated 409 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:31.208167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.273770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:31.284768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '453f0791592c413da9a2399a61c7ec38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:31.306388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '453f0791592c413da9a2399a61c7ec38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:31.307413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '453f0791592c413da9a2399a61c7ec38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:31.308485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '453f0791592c413da9a2399a61c7ec38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:31.330524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-90b22fd8-a858-4bcb-9528-0b36fbb63713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '453f0791592c413da9a2399a61c7ec38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:31.373983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 866/1735] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:31 2022] DELETE /identity/v3/users/453f0791592c413da9a2399a61c7ec38 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:31.391110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.391690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.391975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.396256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3cf08-1408-48aa-b51c-d151736125a3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.420591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:31.421860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:31.559973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.560660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.563760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.563912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 867/1736] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:31.564004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.564004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.564004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.571400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.573221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:31.592266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 870/1737] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:31 2022] POST /identity/v3/users => generated 311 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:31.593347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.596140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:31.606062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:31.614693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4a218-a8df-49b4-874e-d76078b83074 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.628107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.628851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.629160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.629456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.630319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.632648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa7836-cb50-4f09-924a-669f37ddf0ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.647181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 871/1738] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:31.647181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.647181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.647181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.669206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbacfb-8eff-4988-86e0-a36e475df014 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.673736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.674339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.674623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.674900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.676841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] MFA Rules not processed for user `ecf0e8a3fae04b499310fe0d35c08269`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:31.678159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:31.685613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.687424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.702824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 868/1739] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:31.703681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.736333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539c72f-4c69-4fa2-8056-b3d33e162c04 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:31.741513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:31.742301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:31.742686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:31.743060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] PATH_INFO: `/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b/users/cc339fffe8be48fcb7cc6be81c07e1a5/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:31.781962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17692e76eaeb4107b17ebe5e43ebee4b, user_id=cc339fffe8be48fcb7cc6be81c07e1a5, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:31.784956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:31.789505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 872/1740] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:31.828911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 869/1741] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:31 2022] PUT /identity/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b/users/cc339fffe8be48fcb7cc6be81c07e1a5/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:31.832748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:31.846415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba77b-19b1-48b5-9ef7-fdfcf0c57c7e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:32.018247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:32.048103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:32.048841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.049203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.049616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.050354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:32.058959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1748db-8726-49f1-92eb-4ca33e5a9884 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:32.092098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 873/1742] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:32.096090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:32.106941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d4447-57e3-478d-8b47-ce1322199c02 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:32.134084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:32.134773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.135058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.135365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] PATH_INFO: `/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b/users/cc339fffe8be48fcb7cc6be81c07e1a5/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.143845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17692e76eaeb4107b17ebe5e43ebee4b, user_id=cc339fffe8be48fcb7cc6be81c07e1a5, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:32.146617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:32.191288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 870/1743] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:32 2022] PUT /identity/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b/users/cc339fffe8be48fcb7cc6be81c07e1a5/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:32.205834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311616-4832-4966-90c2-7e417a5d818f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.206458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311616-4832-4966-90c2-7e417a5d818f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.206906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311616-4832-4966-90c2-7e417a5d818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.271065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-97311616-4832-4966-90c2-7e417a5d818f None None] MFA Rules not processed for user `cc339fffe8be48fcb7cc6be81c07e1a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:32.275331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97311616-4832-4966-90c2-7e417a5d818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:32.425944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:32.428892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:32.495691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 874/1744] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:32.510454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311616-4832-4966-90c2-7e417a5d818f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:32.512262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97311616-4832-4966-90c2-7e417a5d818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:32.567389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f758-905a-4c51-ad24-7e6440290847 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:32.572987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:32.573635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.573926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.574204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] PATH_INFO: `/v3/projects/1437e428c0ef4f7daa655211a73c9664` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.575402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1437e428c0ef4f7daa655211a73c9664)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:32.577404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:32.589824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311616-4832-4966-90c2-7e417a5d818f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:32.594630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:32.595486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.596260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.596615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.608740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:32.610743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb352a-3990-4751-a025-7754dbf1d464 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:32.637371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 875/1745] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:32.660186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1437e428c0ef4f7daa655211a73c9664', 'request_id': 'req-5711047c-4af5-44d2-994d-ae964174e5b3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:32.674479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1437e428c0ef4f7daa655211a73c9664', 'request_id': 'req-5711047c-4af5-44d2-994d-ae964174e5b3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:32.691049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1437e428c0ef4f7daa655211a73c9664', 'request_id': 'req-5711047c-4af5-44d2-994d-ae964174e5b3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:32.722790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] The token cache is being invalidate because project 1437e428c0ef4f7daa655211a73c9664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:32.722790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:32.722790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5711047c-4af5-44d2-994d-ae964174e5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:32.733975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 871/1746] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:32 2022] DELETE /identity/v3/projects/1437e428c0ef4f7daa655211a73c9664 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:32.738927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.739249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.739562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.768295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:32.769522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:32.855010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 876/1747] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:32.863618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:32.863908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:32.864191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:32.948950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:32.950221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:32.990803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 872/1748] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:32.994842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:33.010618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5e3e9-aa77-41e4-ad2a-97e2dc06b761 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:33.025263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:33.026303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.026805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.027322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:33.028427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:33.029025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:33.032484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:33.067148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 877/1749] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:33.071869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:33.077137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:33.084930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce415c60-056b-467e-87dd-c10f686bfa18 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:33.088873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:33.093436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:33.109345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:33.114403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:33.116038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.116502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.118974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:33.119920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:33.123246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:33.251557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 873/1750] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:33 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:33.253100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:33.262964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b00f6a-4647-4976-98fd-f8d57a585fc5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:33.500643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:33.501475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.504961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.505539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:33.512345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:33.595607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb6f7-6dda-4f78-b866-3ca1aa06e2c5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:33.609666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:33.610320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.610610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.610870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:33.614220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:33.619869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae3339-09b9-4a3e-91a7-95fdaf81d7af admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:33.708196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 874/1751] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:33 2022] POST /identity/v3/users => generated 284 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:33.710207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.710547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.711394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:33.761531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:33.765800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:33.768695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:33.771839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:33.860130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 878/1752] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:33 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 772 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:33.872238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:33.947042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cc94d-b490-4ba8-adde-9acd7fd8e75c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:33.958296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 875/1753] 10.222.0.48 () {58 vars in 1015 bytes} [Tue Aug 30 16:17:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:33.986709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:33.994065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:33.995897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:33.998767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.000720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:34.016595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:34.018564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:34.020946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:34.047746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fff6c2-8ae8-4f3e-85d0-13998fefe81a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:34.074585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:34.078919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:34.079678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:34.081176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.091901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 879/1754] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:34.112074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:34.115895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:34.118731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:34.166396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b454fb1-460b-4f21-863f-deefd24fcd08 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:34.174153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 876/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:34.184760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:34.185975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:34.186239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:34.186523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.187188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:34.194538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb9a3b-f8ff-41db-bbe7-96c4d73e0578 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:34.225244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 880/1756] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:34.227622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:34.248376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f5256-3e06-4d75-b085-f5bbab3743b0 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:34.270372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:34.290862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:34.291158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:34.291460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] PATH_INFO: `/v3/projects/6f4824e7fc74444ea323a8f5174eb999/users/2ee1c8672d034e30b1d65074374eddda/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.332536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4824e7fc74444ea323a8f5174eb999, user_id=2ee1c8672d034e30b1d65074374eddda, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:34.336555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:34.389109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 877/1757] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:34 2022] PUT /identity/v3/projects/6f4824e7fc74444ea323a8f5174eb999/users/2ee1c8672d034e30b1d65074374eddda/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:34.390842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:34.391318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:34.391530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.510634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] MFA Rules not processed for user `2ee1c8672d034e30b1d65074374eddda`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:34.515005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:34.691545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:34.735619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 881/1758] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:34.740749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:34.902324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:34.969102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1f6f1-486e-41a0-bf03-a6f0be16de45 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:34.984052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:34.985018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:34.986869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:34.988459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:34.991146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:34.997291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.072733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 878/1759] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:34 2022] POST /identity/v3/projects => generated 367 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:35.076615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:35.096015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9857d3-7531-488f-a48b-663e482694e2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:35.294826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:35.296407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.296978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.297689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:35.299777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:35.301963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.394977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:35.396924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 882/1760] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:35 2022] POST /identity/v3/users => generated 290 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:35.432874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d8f7-00b1-4017-bee3-2ebd32421b17 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:35.460552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:35.461369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.461950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.462386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:35.463245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:35.465208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.481011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 879/1761] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:35.485664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:35.496355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322c8d-58fe-4f78-876a-543e0e23a611 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:35.511048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:35.512172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.512578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.513132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] PATH_INFO: `/v3/projects/3c7b7f4f375b483e893df9b8c951c04b/users/1f21b3a0ebbb4a1880adff457025707b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:35.564966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7b7f4f375b483e893df9b8c951c04b, user_id=1f21b3a0ebbb4a1880adff457025707b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:35.573615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.665706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 883/1762] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:35 2022] PUT /identity/v3/projects/3c7b7f4f375b483e893df9b8c951c04b/users/1f21b3a0ebbb4a1880adff457025707b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:35.670720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:35.682955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb496c1-0568-40b5-802f-517e473aa223 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:35.812090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:35.813071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.813562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.814308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:35.816316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:35.818300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27abc1-2bb1-4de6-bb7a-3c08d250f8c0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.838908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 880/1763] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:35.843497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:35.875599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c7372-7e3d-4d50-9012-7e8d5a4e6b37 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:35.889586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:35.890391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.890673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.891049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] PATH_INFO: `/v3/projects/3c7b7f4f375b483e893df9b8c951c04b/users/1f21b3a0ebbb4a1880adff457025707b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:35.899121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7b7f4f375b483e893df9b8c951c04b, user_id=1f21b3a0ebbb4a1880adff457025707b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:35.903025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:35.934045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 884/1764] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:35 2022] PUT /identity/v3/projects/3c7b7f4f375b483e893df9b8c951c04b/users/1f21b3a0ebbb4a1880adff457025707b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:35.948199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:35.948800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:35.949683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:36.008828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] MFA Rules not processed for user `1f21b3a0ebbb4a1880adff457025707b`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:36.013182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:36.226196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 881/1765] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:36.440467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:36.634879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b14d08-c677-4689-a47e-fc167818b836 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:36.651123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:36.652638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:36.658642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:36.659101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:36.717152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:36.720073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762154d6-0024-4649-a1cd-d19bb5753ac9 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:36.745773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 885/1766] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:37.099057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:37.103358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:37.128614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ce5e5-b921-482e-92cd-cc643041808e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:37.136551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:37.137518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:37.137963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:37.138423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:37.200130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:37.271752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:37.273235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdc010-d442-4e69-b64f-15e35a40848a service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:37.311208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 882/1767] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:37.786963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:37.787630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:37.787630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:37.789038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 886/1768] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:37.810784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:37.811368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:37.811842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:37.846841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:37.849103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:38.031145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 883/1769] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:17:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:38.090534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:38.093287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:38.130527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c3dfc-1fad-4837-95e7-16e74727feb9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:38.140886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:38.141739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:38.142183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:38.142632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:38.152593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:38.155902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f276a2-e1f8-4024-917a-7d88d6ce0a46 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:38.172957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 887/1770] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:42.797644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:42.799656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:42.934880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35f344-7666-494e-958b-6923de0abef2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:42.937004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:42.937676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:42.937972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:42.938252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] PATH_INFO: `/v3/users/72e7c43452de4c50a2b133ba54c2060b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:42.939564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e7c43452de4c50a2b133ba54c2060b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:42.941553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:43.005016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e7c43452de4c50a2b133ba54c2060b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:43.013249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e7c43452de4c50a2b133ba54c2060b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:43.014035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e7c43452de4c50a2b133ba54c2060b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:43.015270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e7c43452de4c50a2b133ba54c2060b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:43.027166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e7c43452de4c50a2b133ba54c2060b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:43.035744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:43.056379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 884/1771] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:42 2022] DELETE /identity/v3/users/72e7c43452de4c50a2b133ba54c2060b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:43.061397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:43.061981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:43.061981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:43.085587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:43.086829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:43.120277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:43.199090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 888/1772] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:43.209329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:43.210816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:43.288862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc26b-719b-42e7-8fec-6e993d5b5827 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:43.293432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:43.294071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:43.294389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:43.294678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:43.306340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:43.308080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a69ed3-b6e3-4ec5-abe0-de3ebc3ac808 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:43.331321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 885/1773] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:43.813357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:43.818104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:43.842970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ede063-03d6-42ee-8041-e8054c3e955e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:43.859344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:43.860319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:43.860743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:43.861110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] PATH_INFO: `/v3/users/cdc7a21cc31145be8715c4a7e3c085a0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:43.862453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc7a21cc31145be8715c4a7e3c085a0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:43.867437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:43.990888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc7a21cc31145be8715c4a7e3c085a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.004086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc7a21cc31145be8715c4a7e3c085a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.005115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc7a21cc31145be8715c4a7e3c085a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.018084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc7a21cc31145be8715c4a7e3c085a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.038984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc7a21cc31145be8715c4a7e3c085a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.066536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 889/1774] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:43 2022] DELETE /identity/v3/users/cdc7a21cc31145be8715c4a7e3c085a0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:44.073134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:44.073734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:44.073734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:44.113211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:44.114829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:44.120790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:44.123840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:44.253945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 886/1775] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:44.260765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:44.262318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:44.274664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027300-80ad-4ae7-bb59-5d26a43fabe9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:44.288729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:44.289560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:44.289830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:44.290089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] PATH_INFO: `/v3/projects/e594f292f8a148609ecbe4f1ea96c570` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:44.292030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e594f292f8a148609ecbe4f1ea96c570)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:44.298451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:44.329118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd569b-8741-4511-b6ce-8f7f3595a772 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:44.334098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:44.335020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:44.335350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:44.335642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:44.379615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:44.381112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:44.419364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 887/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:44.420978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:44.438493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e594f292f8a148609ecbe4f1ea96c570', 'request_id': 'req-91f899f9-6198-4f0d-b771-014aefa3ee0a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.460126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e594f292f8a148609ecbe4f1ea96c570', 'request_id': 'req-91f899f9-6198-4f0d-b771-014aefa3ee0a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.469470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e594f292f8a148609ecbe4f1ea96c570', 'request_id': 'req-91f899f9-6198-4f0d-b771-014aefa3ee0a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.490594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] The token cache is being invalidate because project e594f292f8a148609ecbe4f1ea96c570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:44.491344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.493471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.496865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 890/1777] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:44 2022] DELETE /identity/v3/projects/e594f292f8a148609ecbe4f1ea96c570 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:44.501637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:44.518530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f14667-cc41-4e7f-bc4a-c8e26e91a5c0 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:44.594095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:44.663533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:44.665873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:44.667662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:44.668429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] PATH_INFO: `/v3/users/7cf3c1e5c3194527b1a637ae3007993f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:44.673077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf3c1e5c3194527b1a637ae3007993f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:44.677178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:44.789854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf3c1e5c3194527b1a637ae3007993f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.797876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf3c1e5c3194527b1a637ae3007993f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.798554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf3c1e5c3194527b1a637ae3007993f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.799323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf3c1e5c3194527b1a637ae3007993f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.808779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf3c1e5c3194527b1a637ae3007993f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:44.838780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 888/1778] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:44 2022] DELETE /identity/v3/users/7cf3c1e5c3194527b1a637ae3007993f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:45.250682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:45.253934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:45.319994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f899f9-6198-4f0d-b771-014aefa3ee0a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:45.498093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:45.499029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:45.499503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:45.500199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] PATH_INFO: `/v3/projects/fb75738585494f908021bd07549a2607` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:45.501629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb75738585494f908021bd07549a2607)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:45.503812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:45.548738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:45.550278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:45.577329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb75738585494f908021bd07549a2607', 'request_id': 'req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:45.582967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30290f-6257-4c2b-8391-17773d01fdd0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:45.589735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb75738585494f908021bd07549a2607', 'request_id': 'req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:45.619678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb75738585494f908021bd07549a2607', 'request_id': 'req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:45.656560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] The token cache is being invalidate because project fb75738585494f908021bd07549a2607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:45.657233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:45.659001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:45.663213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 891/1779] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:45 2022] DELETE /identity/v3/projects/fb75738585494f908021bd07549a2607 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:45.667351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:45.670881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:45.855584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:45.858117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:45.859958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:45.860701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] PATH_INFO: `/v3/users/795a62ad376b49c3bbffa3a302c764a2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:45.864715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=795a62ad376b49c3bbffa3a302c764a2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:45.873069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:45.928912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:45.941280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae26b-a86d-4ec2-96ec-437cb11eb74f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:45.958937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:45.961479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:45.961780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:45.962701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] PATH_INFO: `/v3/users/3c8efff2dd9a4899bdcbdd5292856d63` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:45.965238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8efff2dd9a4899bdcbdd5292856d63)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:45.972300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:46.088128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795a62ad376b49c3bbffa3a302c764a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.115133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:46.122615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795a62ad376b49c3bbffa3a302c764a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.128164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795a62ad376b49c3bbffa3a302c764a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.132581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795a62ad376b49c3bbffa3a302c764a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.156087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37ef1338-8a27-423c-8a94-b0b5146ecef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795a62ad376b49c3bbffa3a302c764a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.185463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8efff2dd9a4899bdcbdd5292856d63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.213764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8efff2dd9a4899bdcbdd5292856d63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.216667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 889/1780] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:45 2022] DELETE /identity/v3/users/795a62ad376b49c3bbffa3a302c764a2 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:46.219814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8efff2dd9a4899bdcbdd5292856d63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.221560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:46.225259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:46.227051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:46.240494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8efff2dd9a4899bdcbdd5292856d63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.293583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8efff2dd9a4899bdcbdd5292856d63'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.309613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 892/1781] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:45 2022] DELETE /identity/v3/users/3c8efff2dd9a4899bdcbdd5292856d63 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:46.317480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:46.320993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:46.432987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 890/1782] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:46.489425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:46.491759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:46.592028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:46.595404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:46.615787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fe0c-5e61-4e57-8e45-3c276d8ebe91 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:46.629805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:46.631452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:46.631595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a08c39-5340-4012-b83f-f0d9dbd0d279 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:46.632018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:46.632443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] PATH_INFO: `/v3/projects/a615235005714840bbbaafac369ddc30` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:46.633828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a615235005714840bbbaafac369ddc30)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:46.637231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:46.644648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:46.646049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:46.646639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:46.647414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:46.683982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:46.713873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d0bd-b37e-4743-8148-8d30928d5bb8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:46.759393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 893/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:46.775958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a615235005714840bbbaafac369ddc30', 'request_id': 'req-8755e382-b527-4b42-8e32-8485feef5d6d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.784255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a615235005714840bbbaafac369ddc30', 'request_id': 'req-8755e382-b527-4b42-8e32-8485feef5d6d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.792795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a615235005714840bbbaafac369ddc30', 'request_id': 'req-8755e382-b527-4b42-8e32-8485feef5d6d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.811663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] The token cache is being invalidate because project a615235005714840bbbaafac369ddc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:46.811991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.812695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8755e382-b527-4b42-8e32-8485feef5d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:46.814651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 891/1784] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:46 2022] DELETE /identity/v3/projects/a615235005714840bbbaafac369ddc30 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:46.820031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:46.820608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:46.821056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:46.847180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:46.848631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:46.925641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 894/1785] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:46.935909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:46.936138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:46.936655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:46.961037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:46.962379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:46.976075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 892/1786] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:46.980858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:46.995683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ab25a-77f3-41e8-b224-2a555330cdfb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.005475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.006261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.006659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.007045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.007855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:47.008350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.010464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.030024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 895/1787] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:47.031679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.043272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b0ef-db20-4d1b-86f0-1e9868908889 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.049745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.050626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.051054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.051631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.052644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.055313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.099090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 893/1788] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:47.099978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.110936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eecb67-326e-4c53-a032-01c66733b390 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.119257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:47.226422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.227236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.227633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.228733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.229614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.232880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ba712-b389-4350-b208-5352799f89f5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.272791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 896/1789] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:47.276237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.293312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ab31b-2050-41ff-a8e4-f49514d331c7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.301168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.302103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.302390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.302665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.303491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.305278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.309175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ffe6-64ed-4366-a4fb-bec1a4df45bb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.310289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ffe6-64ed-4366-a4fb-bec1a4df45bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.310626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ffe6-64ed-4366-a4fb-bec1a4df45bb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.312690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 897/1790] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:17:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:47.335590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.336818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.337283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.348359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 894/1791] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:47.355003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.379474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52656fc-dd73-4be9-b11c-e5b6369f9048 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.379800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:47.382177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.391949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.393117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.393597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.393987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.395534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.398920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.414746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 895/1792] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:47.415801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.419463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.437895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:47.447037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92d7f-1fb0-4657-b404-d42e116a4ce3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.461229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.461998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.462341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.462680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] PATH_INFO: `/v3/users/2ee1c8672d034e30b1d65074374eddda` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.464499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee1c8672d034e30b1d65074374eddda)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.467544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.549924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee1c8672d034e30b1d65074374eddda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.551021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 898/1793] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:47.561569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee1c8672d034e30b1d65074374eddda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.562408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.562622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee1c8672d034e30b1d65074374eddda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.563407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee1c8672d034e30b1d65074374eddda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.572640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee1c8672d034e30b1d65074374eddda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.573971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767face1-8a4f-4968-ab80-33b2b8f5ba5b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.580172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.581404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.582215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.582559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] PATH_INFO: `/v3/projects/7e3d22564a5844ff9b1ab38d2e15b7b2/users/b8117d850b11472c99f8e702a43096aa/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.599026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3d22564a5844ff9b1ab38d2e15b7b2, user_id=b8117d850b11472c99f8e702a43096aa, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.601875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.611432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 896/1794] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:47 2022] DELETE /identity/v3/users/2ee1c8672d034e30b1d65074374eddda => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:47.612954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.615119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.618690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 899/1795] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:47 2022] PUT /identity/v3/projects/7e3d22564a5844ff9b1ab38d2e15b7b2/users/b8117d850b11472c99f8e702a43096aa/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:47.629745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.633440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.711044 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b166d-eac2-48f9-94cd-6b11318d0fc2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.721953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:47.729408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4e863-be6e-497f-ab56-3434d34f4571 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:47.736609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.737346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.737740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.738118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.740238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.758798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.760474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70024a68-57be-48b7-ac8c-3e72b4941f37 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.774204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 900/1796] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:47.775656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6cb1-2197-4894-8aa4-580b1c847647 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.776084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6cb1-2197-4894-8aa4-580b1c847647 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.776450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6cb1-2197-4894-8aa4-580b1c847647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.800836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-00df6cb1-2197-4894-8aa4-580b1c847647 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:47.802140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00df6cb1-2197-4894-8aa4-580b1c847647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.806078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:47.806905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.807279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.807661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] PATH_INFO: `/v3/projects/97ff2cf14a6b407b8825ae28b9985b64` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.809231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ff2cf14a6b407b8825ae28b9985b64)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:47.813019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:47.813181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 901/1797] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:47.814508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:47.814867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:47.815243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:47.847824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] MFA Rules not processed for user `b8117d850b11472c99f8e702a43096aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:47.849942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.900063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:47.933143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ff2cf14a6b407b8825ae28b9985b64', 'request_id': 'req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.941475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ff2cf14a6b407b8825ae28b9985b64', 'request_id': 'req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.949784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ff2cf14a6b407b8825ae28b9985b64', 'request_id': 'req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.960491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 902/1798] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:47.965337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.967501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:47.970295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] The token cache is being invalidate because project 97ff2cf14a6b407b8825ae28b9985b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:47.971032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.972534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:47.977527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 897/1799] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:47 2022] DELETE /identity/v3/projects/97ff2cf14a6b407b8825ae28b9985b64 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:47.978593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:47.981597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.047915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e04f5-8f8c-4f6d-9f11-b62b70795986 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.116141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.116916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.117301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.117659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.119684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.129917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7415-449c-40e6-9fb4-8d3c03c6e0a6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.135464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.137126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b3cff-4785-42ff-ac1d-ccf93267bc50 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.148038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.148674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.148958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 903/1800] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:48.149151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.149338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.149870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.150635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab78ae-3335-4cf1-9318-34f519da6182 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.151066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab78ae-3335-4cf1-9318-34f519da6182 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.151499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab78ae-3335-4cf1-9318-34f519da6182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.151841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.182407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-56ab78ae-3335-4cf1-9318-34f519da6182 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:48.184005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56ab78ae-3335-4cf1-9318-34f519da6182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.191210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 898/1801] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:47 2022] POST /identity/v3/projects => generated 367 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:48.195355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 904/1802] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:48.202628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.213034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.213427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.214426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.247916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:48.247916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.279622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:48.290977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249b3d-593b-457d-8fe6-7a70b4dd6536 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.299751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.300382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.300767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.301108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.301853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.303842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.337727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 899/1803] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:48 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:48.339192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.344102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.345066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 905/1804] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:48.345520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.358633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcbe04-a4b3-401c-8cbf-9605723465eb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.361152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4704a-b786-4b03-bf0b-8d3f20e5b41c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.364647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.365270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.365552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.366017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.366810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.367853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.369012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.369554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.370189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] PATH_INFO: `/v3/projects/ad2d2a9a44bc4ac2a76848f27309fea4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.371680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2d2a9a44bc4ac2a76848f27309fea4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.371998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.374696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.385030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 906/1805] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:48.386115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.398974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9943bb-c103-4769-b0f6-5815b1483b40 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.407285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.407911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.408190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.409156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:48.409889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.410859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:48.411399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.413213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.429982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 907/1806] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:48.430952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.444302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10f48-a259-4e1e-b07f-13aa2d5b4212 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.473486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2d2a9a44bc4ac2a76848f27309fea4', 'request_id': 'req-4d379013-ec1b-4941-b073-6f93d8f38856'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.488411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2d2a9a44bc4ac2a76848f27309fea4', 'request_id': 'req-4d379013-ec1b-4941-b073-6f93d8f38856'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.496851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2d2a9a44bc4ac2a76848f27309fea4', 'request_id': 'req-4d379013-ec1b-4941-b073-6f93d8f38856'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.518056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] The token cache is being invalidate because project ad2d2a9a44bc4ac2a76848f27309fea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:48.518516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.518978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.519256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.519922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.520837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.521375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 900/1807] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:48 2022] DELETE /identity/v3/projects/ad2d2a9a44bc4ac2a76848f27309fea4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:48.521790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] PATH_INFO: `/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244/users/a63623adc5ad45a5b72eb44a5afae948/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.522238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.523765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.538884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379013-ec1b-4941-b073-6f93d8f38856 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.544497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.545979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.545979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.545979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.546343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.548387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.563528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48fca5a56ea4dbcaeb3c3410278b244, user_id=a63623adc5ad45a5b72eb44a5afae948, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.571322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.595827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 901/1808] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:48 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:48.596842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.601125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 908/1809] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:48 2022] PUT /identity/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244/users/a63623adc5ad45a5b72eb44a5afae948/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:48.603975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.604491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.702024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ce46-9174-4a08-bb31-1edf74d27c8c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.749296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:48.763978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7e5-ea44-4d28-9dbd-7cdcdc40c9ae admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.777759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.778909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.780072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.781550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] PATH_INFO: `/v3/users/cc339fffe8be48fcb7cc6be81c07e1a5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.784231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc339fffe8be48fcb7cc6be81c07e1a5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.788676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.797683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.798292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.798565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.799151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.799954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.802155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.834164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 909/1810] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:48 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:48.835203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.837656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.858924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f85eb-9b34-4d4b-879f-cc079cd1c892 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.865681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.866512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.866808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.867085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.867967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.870142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:48.877258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc339fffe8be48fcb7cc6be81c07e1a5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.888375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc339fffe8be48fcb7cc6be81c07e1a5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.889126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 910/1811] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:48.889704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc339fffe8be48fcb7cc6be81c07e1a5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.890041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.890813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc339fffe8be48fcb7cc6be81c07e1a5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.892354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.915838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc339fffe8be48fcb7cc6be81c07e1a5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:48.928897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5455-9bca-4864-883c-300135495c0e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:48.943776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 902/1812] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:48 2022] DELETE /identity/v3/users/cc339fffe8be48fcb7cc6be81c07e1a5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:48.943776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:48.945825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:48.965333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:48.970144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:48.977938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:48.980060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:48.980491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:48.990902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:48.992942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.019428 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 911/1813] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:49.024094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.043155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1fda8-f41d-432b-9146-f8085c22127a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.071113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270330-ab24-4d82-84c6-a71d172096e6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.078707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.079969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.080407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.080956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] PATH_INFO: `/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244/users/a63623adc5ad45a5b72eb44a5afae948/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.084147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48fca5a56ea4dbcaeb3c3410278b244, user_id=a63623adc5ad45a5b72eb44a5afae948, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.087121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df602447-40a0-4ce6-a999-fa178f2e2708 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.113066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 903/1814] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:48 2022] PUT /identity/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244/users/a63623adc5ad45a5b72eb44a5afae948/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:49.125741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.126077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.126307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.171604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.172393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.172771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.173145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.173994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.176717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.181443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] MFA Rules not processed for user `a63623adc5ad45a5b72eb44a5afae948`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:49.183571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.192156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 912/1815] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:49.196120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.212195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25b95-00b4-44b3-b3d4-fdabe7ba8e43 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.315657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.316331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.316609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.316883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] PATH_INFO: `/v3/projects/b5a3aa5bc39c4310ade96cf25c7eedb5/users/d0d39ea152ad4e1db5f30fe89436eb32/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.330923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 904/1816] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:49.331945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.333401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.337431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a3aa5bc39c4310ade96cf25c7eedb5, user_id=d0d39ea152ad4e1db5f30fe89436eb32, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.341508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.363301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fe2d3-8858-4317-9fb7-289696ea3d2b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.365528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 913/1817] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:49 2022] PUT /identity/v3/projects/b5a3aa5bc39c4310ade96cf25c7eedb5/users/d0d39ea152ad4e1db5f30fe89436eb32/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:49.366571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.376111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.422092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:49.490137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.490817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.491670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.491956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] PATH_INFO: `/v3/projects/6f4824e7fc74444ea323a8f5174eb999` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.492364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f8f0-d705-4676-bfd4-e58a7d774a2f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.493186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4824e7fc74444ea323a8f5174eb999)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.495566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.496772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.498056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.498752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.498752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.545682 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:49.573874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4824e7fc74444ea323a8f5174eb999', 'request_id': 'req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.582771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4824e7fc74444ea323a8f5174eb999', 'request_id': 'req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.586492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.588013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed51883-fa73-4b01-9c4f-f03ab5d50580 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.590228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4824e7fc74444ea323a8f5174eb999', 'request_id': 'req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.599200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 914/1818] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:49.600443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.600727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.601090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.614375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] The token cache is being invalidate because project 6f4824e7fc74444ea323a8f5174eb999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:49.614375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.615062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.617659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 905/1819] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:49 2022] DELETE /identity/v3/projects/6f4824e7fc74444ea323a8f5174eb999 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:49.621647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.623290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.656218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] MFA Rules not processed for user `d0d39ea152ad4e1db5f30fe89436eb32`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:49.657482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.728362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72ca0-a4c2-4e80-97d5-092a179ed8f4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.735590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.736779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.738909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.739330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] PATH_INFO: `/v3/users/1f21b3a0ebbb4a1880adff457025707b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.740986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f21b3a0ebbb4a1880adff457025707b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.748067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.826212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 915/1820] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:49.827613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.829912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.864458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f21b3a0ebbb4a1880adff457025707b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.876625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd02480-b766-471e-b53d-298e0d3b3c71 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:49.879988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f21b3a0ebbb4a1880adff457025707b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.880788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f21b3a0ebbb4a1880adff457025707b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.881422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f21b3a0ebbb4a1880adff457025707b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.885134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:49.885780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:49.886060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:49.886330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:49.887475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:49.893279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:49.900685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f21b3a0ebbb4a1880adff457025707b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:49.936951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 906/1821] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:49 2022] DELETE /identity/v3/users/1f21b3a0ebbb4a1880adff457025707b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:49.941285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.943562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:49.986571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 916/1822] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:49 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:49.987685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:49.995453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.015382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:50.103859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:50.118265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549cb6e-3556-4fc0-811e-3ee11a102ebf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.182390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3cfe-cbc7-4e94-9b0b-6cae59c748a7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.193561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.194206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.194667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.194980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.195990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.202948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.241125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 917/1823] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:49 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:50.244852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.251160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.252227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.252731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.253266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] PATH_INFO: `/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.256039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17692e76eaeb4107b17ebe5e43ebee4b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.258128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.264339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c018a6-873a-47a1-90d1-202ba462f4c6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.289326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.290403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.290899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.291397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.293530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.296013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.331695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 918/1824] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:50.331695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.346731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310dd96-ca9d-4a54-bc8d-d74466926f8c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.351364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17692e76eaeb4107b17ebe5e43ebee4b', 'request_id': 'req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.361961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17692e76eaeb4107b17ebe5e43ebee4b', 'request_id': 'req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.370334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17692e76eaeb4107b17ebe5e43ebee4b', 'request_id': 'req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.394201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] The token cache is being invalidate because project 17692e76eaeb4107b17ebe5e43ebee4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:50.394688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.395650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f0255129-ab9c-4c8d-b4e1-ead6b9f64e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.398056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 907/1825] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:49 2022] DELETE /identity/v3/projects/17692e76eaeb4107b17ebe5e43ebee4b => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:50.402836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.411733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.412171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.442526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:50.445394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.470793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.472121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.472415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.472854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] PATH_INFO: `/v3/projects/57de956e082044fdae843824b1c39682/users/55cad95a4d064c0bb1726971c2b7a1c7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.516358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57de956e082044fdae843824b1c39682, user_id=55cad95a4d064c0bb1726971c2b7a1c7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.519382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.543935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 919/1826] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:50 2022] PUT /identity/v3/projects/57de956e082044fdae843824b1c39682/users/55cad95a4d064c0bb1726971c2b7a1c7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:50.549592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.549912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.594025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 908/1827] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:50.595641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.597052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.610511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57008ba-d25c-4af6-9cb7-6548d640da9e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.635781 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:50.675672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd942c34-1ea0-4103-bdd5-38367837af0b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.682080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.682707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.682995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.683316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.684019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.687892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cc5e9-9dcc-45b0-b0bf-c6f0d7504c84 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.701554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 920/1828] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:50.704887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.706043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.706337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.722857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.722857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.722857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.722857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] PATH_INFO: `/v3/projects/3c7b7f4f375b483e893df9b8c951c04b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.729810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:50.731031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.731405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7b7f4f375b483e893df9b8c951c04b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.736594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.741016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 921/1829] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:50.742079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.757140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05c28a-831e-4779-9b10-12bcc7908006 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.763744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.764462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.765019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.765315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] PATH_INFO: `/v3/projects/57de956e082044fdae843824b1c39682/users/55cad95a4d064c0bb1726971c2b7a1c7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.770422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=57de956e082044fdae843824b1c39682, user_id=55cad95a4d064c0bb1726971c2b7a1c7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.775062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.800267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 922/1830] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:50 2022] PUT /identity/v3/projects/57de956e082044fdae843824b1c39682/users/55cad95a4d064c0bb1726971c2b7a1c7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:50.800954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.809652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7b7f4f375b483e893df9b8c951c04b', 'request_id': 'req-364f8c4d-7c2a-4fce-9757-6782822f54f6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.816131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:50.817777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7b7f4f375b483e893df9b8c951c04b', 'request_id': 'req-364f8c4d-7c2a-4fce-9757-6782822f54f6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.825268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7b7f4f375b483e893df9b8c951c04b', 'request_id': 'req-364f8c4d-7c2a-4fce-9757-6782822f54f6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.846751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] The token cache is being invalidate because project 3c7b7f4f375b483e893df9b8c951c04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:50.847053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.848096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-364f8c4d-7c2a-4fce-9757-6782822f54f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:50.849937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 909/1831] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:50 2022] DELETE /identity/v3/projects/3c7b7f4f375b483e893df9b8c951c04b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:50.851052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808366-a40f-44fa-883b-d75b19e01ff8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.851376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808366-a40f-44fa-883b-d75b19e01ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.851649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808366-a40f-44fa-883b-d75b19e01ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.885594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013a5c-39d7-46af-a212-fe1dc2ea14eb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:50.885879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8a808366-a40f-44fa-883b-d75b19e01ff8 None None] MFA Rules not processed for user `55cad95a4d064c0bb1726971c2b7a1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:50.885879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8a808366-a40f-44fa-883b-d75b19e01ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:50.962244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:50.963192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.963583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.963813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.964495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:50.964781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:50.967396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:50.970746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 910/1832] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:50.973086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:50.973762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:50.974239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:50.989654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 923/1833] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:50.990514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:50.994727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.003686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:51.004897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.013830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 911/1834] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:51.014638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.015927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.028758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88456683-e314-4b9a-93e5-6e28c1a22193 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.039042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.039042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.039042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.039042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.039452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.039452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254893b-fb3c-46c2-bb38-cc16c5a87e07 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.082582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 912/1835] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:51.084255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.084550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.084859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.086873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f06295-d225-4138-8c27-755480a9e47a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.096258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:51.121664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:51.124627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.171712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.172360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.172637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.172907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.234722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:51.240358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 913/1836] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:51.241569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.258999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185738-b144-4c7d-b970-c641bcb55776 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.264674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.265299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.265589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.265869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.267159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.269349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.287411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.289633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.302209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 924/1837] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:51.314091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 914/1838] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/users => generated 284 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:51.314442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.322242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.328612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbccc7-5127-4e36-9c56-bbee32706e53 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.334156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.335076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.335507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.335889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.336659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:51.337070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.338906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.345174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24472-a332-4a33-9851-06f671f69222 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.351675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.352370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.352730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.353084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.353702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 925/1839] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:51.353828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.354641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.355635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.360932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.377846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fb93c-43d1-4438-aaac-4359f9b70624 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.387270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.388083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.388667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.388978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] PATH_INFO: `/v3/users/d0d39ea152ad4e1db5f30fe89436eb32` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.393918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d39ea152ad4e1db5f30fe89436eb32)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.394075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 915/1840] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:51.394171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.408886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.432724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a458-62f9-4214-ac2c-d5650b943ec3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.439492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.440117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.440403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.440680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.441403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.443546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.486087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 916/1841] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:51.486087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.487762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d39ea152ad4e1db5f30fe89436eb32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.498397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d39ea152ad4e1db5f30fe89436eb32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.499616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d39ea152ad4e1db5f30fe89436eb32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.513178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d39ea152ad4e1db5f30fe89436eb32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.530760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d39ea152ad4e1db5f30fe89436eb32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.557891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 926/1842] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:51 2022] DELETE /identity/v3/users/d0d39ea152ad4e1db5f30fe89436eb32 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:51.559882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.567651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.633980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd38dd9-4be4-4a31-b5f5-99ca34fa4fab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.722977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.723757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.724948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.725257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.726038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.727785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86672c5c-f8ad-4f97-b444-d9aa8061c1a2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.729299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:51.751194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 917/1843] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:51.752278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.757543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3353df-3005-4ff8-b7b4-7b104656615b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.759049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.760558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.772389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.773016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.773555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.774012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.774843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.777637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.805846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:51.807244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.817227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 918/1844] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:51.818202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.820393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.834759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 927/1845] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:51 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:51.835743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.839827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699399d3-79d2-4f3f-a4ed-db2156fef29d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.842350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:51.846870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.847805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.848391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.848847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] PATH_INFO: `/v3/projects/be67f2c9b37d4698adb20189168aabc9/users/c9a92cee33e6478882754b5b2c176cd2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.867593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=be67f2c9b37d4698adb20189168aabc9, user_id=c9a92cee33e6478882754b5b2c176cd2, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.870572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9246a-4917-4004-a37c-399858951bb9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.870955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.883285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.883955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.884235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.884514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] PATH_INFO: `/v3/users/b8117d850b11472c99f8e702a43096aa` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.892114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 919/1846] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:51 2022] PUT /identity/v3/projects/be67f2c9b37d4698adb20189168aabc9/users/c9a92cee33e6478882754b5b2c176cd2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:51.892731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8117d850b11472c99f8e702a43096aa)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.893176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:51.894237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.902470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:51.971061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8117d850b11472c99f8e702a43096aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.975178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c961c3-33f2-4832-aaf0-6f67649ec9dd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:51.983847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8117d850b11472c99f8e702a43096aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.984030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:51.984750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:51.984917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8117d850b11472c99f8e702a43096aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:51.985224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:51.985503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:51.986330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:51.988314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83148-11a5-46b5-9c00-bed363f38f4c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:51.996234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8117d850b11472c99f8e702a43096aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.009517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 920/1847] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:52.009517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.009517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.009517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.014777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8117d850b11472c99f8e702a43096aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.024800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 928/1848] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:51 2022] DELETE /identity/v3/users/b8117d850b11472c99f8e702a43096aa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:52.024800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.028366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.045342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] MFA Rules not processed for user `c9a92cee33e6478882754b5b2c176cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:52.046582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.103682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9c8d-1a79-4a1d-a22f-5675517fdb77 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.112901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.113630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.114918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.115202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.117625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.154042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 921/1849] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:52.156254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.167484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.211376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.213253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d43eb-fc20-4d97-a3f8-52744772ad29 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.232339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 929/1850] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:52.234049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.234340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.234601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.244460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:52.270495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:52.271712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.281327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 930/1851] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:52.281729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6461e-eb93-48ea-9ea8-9baaf9f8144e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.282171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.283514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.288784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.289569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.289965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.290348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.291173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.293248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.306482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 922/1852] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:52.307619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.311547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.321353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:52.326601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf9ef-1948-41d9-8245-6e614a8e3c88 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.331697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.332515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.332876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.333226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.344661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.346432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.358068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 923/1853] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:52.359030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.370435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0adc0a-f166-4d54-a81f-1c2842bb9a5f service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.385107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.385944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.386234 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.386515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] PATH_INFO: `/v3/projects/65c00ee7d621478daf926f90b1759d60/users/1d05ba0207a542ae81288e99bb5c1791/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.394832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142ea2-3202-4d40-8418-65d5ae449048 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.399603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.400694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.401161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.401638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.407712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c00ee7d621478daf926f90b1759d60, user_id=1d05ba0207a542ae81288e99bb5c1791, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.423046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d6035-3903-4f7f-8708-e26c3e366c33 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.429675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.431618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.439150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 924/1854] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:52 2022] PUT /identity/v3/projects/65c00ee7d621478daf926f90b1759d60/users/1d05ba0207a542ae81288e99bb5c1791/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:52.451565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.451565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.451875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.467156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 931/1855] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:52.491559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.493622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.523392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb794fb6-e975-461e-a33f-3ded00031220 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.525136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] MFA Rules not processed for user `1d05ba0207a542ae81288e99bb5c1791`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:52.526919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.611136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.612162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.612323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.612751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] PATH_INFO: `/v3/users/c9a92cee33e6478882754b5b2c176cd2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.615085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a92cee33e6478882754b5b2c176cd2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.617452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.620934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 925/1856] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:52.626256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.667099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec441ca-9f04-48df-a3e1-5b3567a74c28 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.679108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.679782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.680086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.680397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.680994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.685293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.695108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a92cee33e6478882754b5b2c176cd2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.709273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a92cee33e6478882754b5b2c176cd2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.710756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a92cee33e6478882754b5b2c176cd2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.721636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a92cee33e6478882754b5b2c176cd2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.725608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 926/1857] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:52.729295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.740567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4e0dfbc0-a0af-42ee-a439-92000ba59ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a92cee33e6478882754b5b2c176cd2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:52.748605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 932/1858] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:52 2022] DELETE /identity/v3/users/c9a92cee33e6478882754b5b2c176cd2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:52.753596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.754313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.754727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.785459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:52.787195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.826008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f538d-6d39-4bf9-b041-d7212f926ac1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:52.852666 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:52.877112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:52.910438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 933/1859] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:52.930089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:52.930641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.930763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:52.930839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:52.931163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:52.931880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:52.932198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:52.934278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:52.973702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 927/1860] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:52 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:52.978687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:52.983156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:53.010107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f031ea-04e1-4f3d-b416-0a707bd5524f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.018041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.018895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.019574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.020038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.021199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.023305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.034569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aaa1c-528a-42d1-9e99-d6f9fbc6f90d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.039704 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.044237 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 928/1861] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:53.045325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.045833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.046566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.052650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.062775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.067547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.068691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c456f9-9d47-4bf6-b258-06c3089b15df admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.078780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.081803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.082523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.083147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] PATH_INFO: `/v3/projects/a7c10fd51f68438e96d4d1324baf97af/users/7f8e9666d8554001a70dd4ea9940c3ac/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.086257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 934/1862] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:53.111545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c10fd51f68438e96d4d1324baf97af, user_id=7f8e9666d8554001a70dd4ea9940c3ac, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.115884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.138101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 929/1863] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:53 2022] PUT /identity/v3/projects/a7c10fd51f68438e96d4d1324baf97af/users/7f8e9666d8554001a70dd4ea9940c3ac/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:53.142102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.156863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ac5-5ba6-4110-abf9-43f57048e597 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.280674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.282188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.282331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.282884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.283953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.286045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406dd2b-51f1-4f4d-92f2-a906c5c4efe6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.302015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 935/1864] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:53.303844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.316651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f0c0-b28d-422e-a5a9-1cb0076649ce admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.323770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.324533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.324887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.325172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] PATH_INFO: `/v3/projects/a7c10fd51f68438e96d4d1324baf97af/users/7f8e9666d8554001a70dd4ea9940c3ac/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.328199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c10fd51f68438e96d4d1324baf97af, user_id=7f8e9666d8554001a70dd4ea9940c3ac, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.331475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.363290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 930/1865] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:53 2022] PUT /identity/v3/projects/a7c10fd51f68438e96d4d1324baf97af/users/7f8e9666d8554001a70dd4ea9940c3ac/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:53.375705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.375705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.375705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.424371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] MFA Rules not processed for user `7f8e9666d8554001a70dd4ea9940c3ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:53.427351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:53.535491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 936/1866] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:53.546530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.548180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:53.610362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51719a-3fd3-490d-8f99-cb601da5377b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.615025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.615661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.615945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.616223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.626709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.628028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.639666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 931/1867] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:53.769381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.770909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:53.799148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:53.920877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fad95-5ccb-43b6-a97b-eac82a99f519 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.921707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:53.925886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:53.929581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.930231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.930517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.930791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] PATH_INFO: `/v3/projects/7e3d22564a5844ff9b1ab38d2e15b7b2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.931970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3d22564a5844ff9b1ab38d2e15b7b2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.933897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:53.948654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc753-022a-4def-a379-782941068faa service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:53.956261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:53.957454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:53.957454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:53.957454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] PATH_INFO: `/v3/projects/b5a3aa5bc39c4310ade96cf25c7eedb5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:53.958871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a3aa5bc39c4310ade96cf25c7eedb5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:53.961100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:54.044838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3d22564a5844ff9b1ab38d2e15b7b2', 'request_id': 'req-b7949a6d-4604-4043-8110-31451e068f32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.054991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3d22564a5844ff9b1ab38d2e15b7b2', 'request_id': 'req-b7949a6d-4604-4043-8110-31451e068f32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.063852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a3aa5bc39c4310ade96cf25c7eedb5', 'request_id': 'req-9b13d28b-b395-491b-832a-f1f0258a600a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.064129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3d22564a5844ff9b1ab38d2e15b7b2', 'request_id': 'req-b7949a6d-4604-4043-8110-31451e068f32'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.073353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a3aa5bc39c4310ade96cf25c7eedb5', 'request_id': 'req-9b13d28b-b395-491b-832a-f1f0258a600a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.085298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] The token cache is being invalidate because project 7e3d22564a5844ff9b1ab38d2e15b7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:54.085588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.088031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.092591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 937/1868] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:53 2022] DELETE /identity/v3/projects/7e3d22564a5844ff9b1ab38d2e15b7b2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.093492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:54.098002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a3aa5bc39c4310ade96cf25c7eedb5', 'request_id': 'req-9b13d28b-b395-491b-832a-f1f0258a600a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.098633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.101440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:54.133449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] The token cache is being invalidate because project b5a3aa5bc39c4310ade96cf25c7eedb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:54.133842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.134643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.146595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 932/1869] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:53 2022] DELETE /identity/v3/projects/b5a3aa5bc39c4310ade96cf25c7eedb5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.152965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:54.159806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.287542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7949a6d-4604-4043-8110-31451e068f32 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:54.318763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13d28b-b395-491b-832a-f1f0258a600a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:54.325282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:54.326073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.326435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.326840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] PATH_INFO: `/v3/users/a63623adc5ad45a5b72eb44a5afae948` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.328153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63623adc5ad45a5b72eb44a5afae948)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:54.330245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:54.383953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:54.408223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63623adc5ad45a5b72eb44a5afae948'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.421612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:54.422565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.422859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.423190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] PATH_INFO: `/v3/projects/be67f2c9b37d4698adb20189168aabc9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.423345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63623adc5ad45a5b72eb44a5afae948'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.424131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63623adc5ad45a5b72eb44a5afae948'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.424684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be67f2c9b37d4698adb20189168aabc9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:54.425127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63623adc5ad45a5b72eb44a5afae948'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.426827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:54.438869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63623adc5ad45a5b72eb44a5afae948'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.479656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 933/1870] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:54 2022] DELETE /identity/v3/users/a63623adc5ad45a5b72eb44a5afae948 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.479656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:54.479656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.493307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be67f2c9b37d4698adb20189168aabc9', 'request_id': 'req-5ea43346-0106-48b4-8e0c-155bb467340b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.506486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be67f2c9b37d4698adb20189168aabc9', 'request_id': 'req-5ea43346-0106-48b4-8e0c-155bb467340b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.529667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be67f2c9b37d4698adb20189168aabc9', 'request_id': 'req-5ea43346-0106-48b4-8e0c-155bb467340b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.584797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] The token cache is being invalidate because project be67f2c9b37d4698adb20189168aabc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:54.585461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.586164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5ea43346-0106-48b4-8e0c-155bb467340b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.588217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 938/1871] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:54 2022] DELETE /identity/v3/projects/be67f2c9b37d4698adb20189168aabc9 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.594217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.594661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.595405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.620358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166011ae-9138-4b16-ab06-73579c66131b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:54.632249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:54.632893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.657561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:54.726265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 939/1872] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:54.727381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:54.728726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.742509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106eb3a-1c61-4ce2-9fa1-7db91f2944b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:54.748891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:54.749524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.749807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.751443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] PATH_INFO: `/v3/users/1d05ba0207a542ae81288e99bb5c1791` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.752955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d05ba0207a542ae81288e99bb5c1791)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:54.755396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:54.759990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:54.760793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.761009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.761293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] PATH_INFO: `/v3/users/55cad95a4d064c0bb1726971c2b7a1c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.762633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] RBAC: Authorizing `identity:delete_user(user_id=55cad95a4d064c0bb1726971c2b7a1c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:54.764727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:54.830145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d05ba0207a542ae81288e99bb5c1791'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.840509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d05ba0207a542ae81288e99bb5c1791'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.841570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55cad95a4d064c0bb1726971c2b7a1c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.842022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d05ba0207a542ae81288e99bb5c1791'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.852021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55cad95a4d064c0bb1726971c2b7a1c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.853189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55cad95a4d064c0bb1726971c2b7a1c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.853868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55cad95a4d064c0bb1726971c2b7a1c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.859541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d05ba0207a542ae81288e99bb5c1791'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.865685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0c067cf4-fa4d-4dcb-9e04-06c2220d908a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55cad95a4d064c0bb1726971c2b7a1c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.879415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:54.896887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-433f72ac-e0e9-4d52-bc97-4bd234a0f44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d05ba0207a542ae81288e99bb5c1791'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:54.905137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 934/1873] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:54 2022] DELETE /identity/v3/users/55cad95a4d064c0bb1726971c2b7a1c7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.906575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.906865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.907144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.910108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 940/1874] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:54 2022] DELETE /identity/v3/users/1d05ba0207a542ae81288e99bb5c1791 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:54.916286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:54.916580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:54.916980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:54.934508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:54.936356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:54.948058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:54.950023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.080055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 935/1875] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:55.084642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.096127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 941/1876] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:55.104996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce8c94-f230-4320-adc5-8868b54adcf7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.112174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.113190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.113190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.114096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.114771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:55.115078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.116873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.122225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.124387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.135544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 936/1877] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:55.136745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.152009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22b8c-b23c-477c-bdf4-a00b088bd291 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.160236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.160874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.161347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.161847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.162318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.164403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.218481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 937/1878] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/projects => generated 361 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:55.224036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.230480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:55.243955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16d2a-8144-429f-af90-f74968bc7bf6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.323077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.323727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.324180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.324489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.342932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.344900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111369f-305c-44be-8e31-5ff9283d5153 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.360537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7629f8-15d2-44c5-9391-de98794e9778 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.374273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 942/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:55.378687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.381321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.382410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.386643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.387902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.392257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.482060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 938/1880] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/users => generated 281 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:55.483324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.483888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.484118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.484466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.486519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 943/1881] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:17:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:17:55.507281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546c65f-dfc2-436f-8466-dc885cdfdda4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.509265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.510987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.521447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.523957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.535868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.535868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.536151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.538591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5405c-c4f8-4466-9012-192cf7e566a6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.540818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed89c7-d565-49ae-9047-ed09e3c4b57e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.544176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.545137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.545628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.546112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] PATH_INFO: `/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.548162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48fca5a56ea4dbcaeb3c3410278b244)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.550324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.569850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 939/1882] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:55.571603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5cdb9-d87d-41e7-965c-a3352260b8cf None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.572642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5cdb9-d87d-41e7-965c-a3352260b8cf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.572967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5cdb9-d87d-41e7-965c-a3352260b8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.649003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fdb5cdb9-d87d-41e7-965c-a3352260b8cf None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:55.650869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:55.651439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fdb5cdb9-d87d-41e7-965c-a3352260b8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.657556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48fca5a56ea4dbcaeb3c3410278b244', 'request_id': 'req-ca106ad7-69ee-4838-98d7-a5aaa60d1972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:55.671527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48fca5a56ea4dbcaeb3c3410278b244', 'request_id': 'req-ca106ad7-69ee-4838-98d7-a5aaa60d1972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:55.684520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48fca5a56ea4dbcaeb3c3410278b244', 'request_id': 'req-ca106ad7-69ee-4838-98d7-a5aaa60d1972'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:55.699396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] The token cache is being invalidate because project b48fca5a56ea4dbcaeb3c3410278b244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:55.700706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:55.701392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:55.703379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 944/1883] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:55 2022] DELETE /identity/v3/projects/b48fca5a56ea4dbcaeb3c3410278b244 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:17:55.707775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.709482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.803864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 940/1884] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:55.813384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf99b3-fd67-4bd3-ae5d-1493a6f9c186 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.813769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf99b3-fd67-4bd3-ae5d-1493a6f9c186 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.814082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf99b3-fd67-4bd3-ae5d-1493a6f9c186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.839062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7baf99b3-fd67-4bd3-ae5d-1493a6f9c186 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:55.842034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7baf99b3-fd67-4bd3-ae5d-1493a6f9c186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.852315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 941/1885] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:55.868412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.870219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106ad7-69ee-4838-98d7-a5aaa60d1972 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.871451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.872034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.877360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.878230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.878403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.879300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.879537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.881920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.897593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 945/1886] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:55.904507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.919997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:55.920724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95b3c5-2083-45d3-96c0-1e2c78844779 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.921096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.927514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.928376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.928744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.929192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] PATH_INFO: `/v3/projects/63cd54835b0244feba41764e026213f8/users/a7dee073270a40de8d9851aadbbc6ec1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.931059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 942/1887] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:55.936037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.950655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35355603-1dfc-40d7-8df7-a83d934bd18b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:55.956036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:55.956905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:55.957415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:55.957660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:55.958439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:55.958874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.960627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.971708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cd54835b0244feba41764e026213f8, user_id=a7dee073270a40de8d9851aadbbc6ec1, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:55.974541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:55.976596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 943/1888] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:17:55.978195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:55.982512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:55.998971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 946/1889] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:55 2022] PUT /identity/v3/projects/63cd54835b0244feba41764e026213f8/users/a7dee073270a40de8d9851aadbbc6ec1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.000715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.021382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22caa27-f6fc-433a-93a4-0a0badc3483b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.107513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:56.144384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7da9-e5cf-4694-946f-83ef328ef86f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.151665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.152422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.152736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.152902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.153397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.153519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.153689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.153970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.154674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.157451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291eda78-fe45-4401-b118-63b35d9ffaef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.163641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.165478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.177246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 944/1890] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:56.178305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.180067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.196764 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78446ec-daf2-4076-839d-b2fd220a3848 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.203386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 947/1891] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:55 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:56.204043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.204786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.205205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.205392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.205712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.205837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] PATH_INFO: `/v3/projects/65c00ee7d621478daf926f90b1759d60` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.206274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.207458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c00ee7d621478daf926f90b1759d60)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.209532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.251445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] MFA Rules not processed for user `a7dee073270a40de8d9851aadbbc6ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:56.253234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.277895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c00ee7d621478daf926f90b1759d60', 'request_id': 'req-ecad91e3-2681-466d-8960-49122c1aabc1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.279734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:56.286690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c00ee7d621478daf926f90b1759d60', 'request_id': 'req-ecad91e3-2681-466d-8960-49122c1aabc1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.293007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c00ee7d621478daf926f90b1759d60', 'request_id': 'req-ecad91e3-2681-466d-8960-49122c1aabc1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.305888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] The token cache is being invalidate because project 65c00ee7d621478daf926f90b1759d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:56.306266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.306986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.311724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 945/1892] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:56 2022] DELETE /identity/v3/projects/65c00ee7d621478daf926f90b1759d60 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.311724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.311724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.332476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 948/1893] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:56.333464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.334791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.383883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad91e3-2681-466d-8960-49122c1aabc1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.389592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.390207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.390487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.390760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] PATH_INFO: `/v3/projects/57de956e082044fdae843824b1c39682` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.391912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57de956e082044fdae843824b1c39682)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.393984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.419335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8868-875f-46ad-8f35-3665fe93d524 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.425154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.426014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.426456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.426855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.427831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.430078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.445294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57de956e082044fdae843824b1c39682', 'request_id': 'req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.452694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57de956e082044fdae843824b1c39682', 'request_id': 'req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.455405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:56.457474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 949/1894] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:56 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:56.458542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.460276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.464666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57de956e082044fdae843824b1c39682', 'request_id': 'req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.480054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59f42-2c10-469a-8673-cb7451e2863b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.481419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] The token cache is being invalidate because project 57de956e082044fdae843824b1c39682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:56.482221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.483004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.484779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 946/1895] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:56 2022] DELETE /identity/v3/projects/57de956e082044fdae843824b1c39682 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.486103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.487673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.556465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.557404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.557579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b81d82-2bfa-4ef0-8d63-72d88c38328c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.557984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.558408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] PATH_INFO: `/v3/users/7f8e9666d8554001a70dd4ea9940c3ac` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.559954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8e9666d8554001a70dd4ea9940c3ac)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.561402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.562012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.562294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.562572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.562995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.564566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.630986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8e9666d8554001a70dd4ea9940c3ac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.636004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.637472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.638207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:56.643608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8e9666d8554001a70dd4ea9940c3ac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.645010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8e9666d8554001a70dd4ea9940c3ac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.655504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 947/1896] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:56.655755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8e9666d8554001a70dd4ea9940c3ac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.656322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.658497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.670813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-45d161b4-c8e4-4038-a911-c175a29497ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8e9666d8554001a70dd4ea9940c3ac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.674775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea965f21-e40b-4f2c-b5df-d8c4cf23b424 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.677882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 950/1897] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:56 2022] DELETE /identity/v3/users/7f8e9666d8554001a70dd4ea9940c3ac => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.679546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.679700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.679969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.680344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.680446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.690897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.691536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.693176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.694050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.706727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 948/1898] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:56.708233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.710468 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:56.711463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.712066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.726088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c813a-0589-4d4f-aea2-4fba7130b9b2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.789696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 951/1899] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:56.790562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.791988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.800427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.801380 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.801779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.802198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] PATH_INFO: `/v3/users/a7dee073270a40de8d9851aadbbc6ec1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.803949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7dee073270a40de8d9851aadbbc6ec1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.806427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.806761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caac2f7-247f-4fba-b4e9-15a615c717c0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.813904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.814614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.815074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.815399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.816072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.817847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87396d4d-a34b-4637-8b6f-a8f2337bca55 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.837205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 952/1900] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:56.838773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.839073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.839384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.860361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:56.866219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:56.869022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.881367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 953/1901] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:56.882268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.894788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7dee073270a40de8d9851aadbbc6ec1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.895670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d68da-82c3-43af-a3ed-1e42119060e8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:56.900526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:56.901395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.901782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.902130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] PATH_INFO: `/v3/projects/06c33fcecae44f58a68b42a87984925f/users/9c44c63a58584625a5dd60ccd9addb95/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.903942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7dee073270a40de8d9851aadbbc6ec1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.905083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7dee073270a40de8d9851aadbbc6ec1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.906851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7dee073270a40de8d9851aadbbc6ec1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.916654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6e719108-5233-4b71-9fb4-2e7e99cbf8b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7dee073270a40de8d9851aadbbc6ec1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:56.919840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c33fcecae44f58a68b42a87984925f, user_id=9c44c63a58584625a5dd60ccd9addb95, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:56.922626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:56.947045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 954/1902] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:56 2022] PUT /identity/v3/projects/06c33fcecae44f58a68b42a87984925f/users/9c44c63a58584625a5dd60ccd9addb95/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.947323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 949/1903] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:17:56 2022] DELETE /identity/v3/users/a7dee073270a40de8d9851aadbbc6ec1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:56.950567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:56.952456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:56.952998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be16e-9fda-48cc-b419-d6ade47b6be7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:56.953337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be16e-9fda-48cc-b419-d6ade47b6be7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:56.953697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be16e-9fda-48cc-b419-d6ade47b6be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:56.982139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-151be16e-9fda-48cc-b419-d6ade47b6be7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:56.983907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-151be16e-9fda-48cc-b419-d6ade47b6be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.063639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890474f4-41d8-48e5-91fd-0115e8e5db6a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.109042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:57.154892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.155699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.156065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.156417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.157418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 950/1904] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:57.158007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:57.158565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.158863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.159142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.162085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.165298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.189306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 955/1905] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:57.194473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.205738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:57.208420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.220251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3f3b-35ac-42ca-b51f-81cc2242d79c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.224504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 951/1906] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:57.237565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.239625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.240779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.241094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.241380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.241668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.242343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.244338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.298647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 956/1907] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:17:57.298647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.301538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.357131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc60760-4be6-4c72-95ea-fe4ed84e3a07 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.446520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.447664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.448160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.448778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.454108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0b6e8-0635-4c27-b0ee-168c418b202c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.468168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:57.469953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.471583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.473765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.474062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.578311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.579799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.589918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 952/1908] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:17:57.590492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.590766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.592667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.593522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.605242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266292-e9b7-4c21-bc3a-78fc47110e9e service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.607351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 957/1909] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:57.608450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.610667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.611891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.612278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.612660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] PATH_INFO: `/v3/projects/a7c10fd51f68438e96d4d1324baf97af` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.613919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c10fd51f68438e96d4d1324baf97af)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.616022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.622976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451418-64c5-4e3e-9fcc-8e4a092bc723 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.628802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.629917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.630885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.631563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.632829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.637076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.677359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 958/1910] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/users => generated 295 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:57.678452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.696952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cbd59-242f-4e0d-89df-adb305d94ffa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.710010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c10fd51f68438e96d4d1324baf97af', 'request_id': 'req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:57.721238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c10fd51f68438e96d4d1324baf97af', 'request_id': 'req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:57.735165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c10fd51f68438e96d4d1324baf97af', 'request_id': 'req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:57.750365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:57.757887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] The token cache is being invalidate because project a7c10fd51f68438e96d4d1324baf97af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:57.758555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:57.760740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c1c1fa50-ce39-4a79-a4cb-25d31378b9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:57.763213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 953/1911] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:57 2022] DELETE /identity/v3/projects/a7c10fd51f68438e96d4d1324baf97af => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:57.770164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e011df-9856-47f4-a6e0-bd919fce07c4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.770768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e011df-9856-47f4-a6e0-bd919fce07c4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.771835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e011df-9856-47f4-a6e0-bd919fce07c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.798257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.798847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.799134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.799438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.802017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.803701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.811040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e6e011df-9856-47f4-a6e0-bd919fce07c4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:57.812352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6e011df-9856-47f4-a6e0-bd919fce07c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.824426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 954/1912] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:57.837136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.837731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 959/1913] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:57.838018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.838360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.842063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.844499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.858484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae23-da37-4a08-80eb-befcb2c83d89 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.866192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:57.867423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:57.868275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.868888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.871379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.871744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.876722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.881535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.885455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 955/1914] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:57.889981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.903196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead12c7-f81e-48ac-b018-3b5e2c8c8baf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.904356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 960/1915] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:17:57.908648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.909741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.910371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.910658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.910936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.911645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:57.911950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.914381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.925065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5c864-162d-480a-ae9e-f1f22dd8c2b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.930470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 956/1916] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:17:57.933002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.933978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.935292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.935731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] PATH_INFO: `/v3/projects/94a5609af9f54858b048ce9b7b33052e/users/6b7aa7d9187f42209b91d4473fc6d1a1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.939376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:57.959163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a5609af9f54858b048ce9b7b33052e, user_id=6b7aa7d9187f42209b91d4473fc6d1a1, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.962797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996cb0-e7c9-497f-8d50-9b63860e5a2b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.965247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae2ccd-f1fc-4b5e-b880-e0001ae09a41 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:57.970330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:57.970941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.971247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.971532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:57.973235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:57.975311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:57.987454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 961/1917] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:57 2022] PUT /identity/v3/projects/94a5609af9f54858b048ce9b7b33052e/users/6b7aa7d9187f42209b91d4473fc6d1a1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:57.991679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:57.992053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:57.992335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.021356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 957/1918] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:58.025305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.030068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] MFA Rules not processed for user `6b7aa7d9187f42209b91d4473fc6d1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:58.031589 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.086387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:58.110064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 962/1919] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:17:58.111631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.113523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f15953-8189-48e6-9d9a-1091347438f6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.122766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.123601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.123914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.124192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.124840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.127362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86541b-c35f-423d-8bec-407ee3b885c1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.128102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.133628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.134498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.134949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.135434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.136250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.139513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.189235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 963/1920] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:58.193440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 958/1921] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:58.195517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.195815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.197389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.214215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677870-1b77-4cd4-b556-cf47fe0e174c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.327400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b773a-c688-4b10-90fb-7e27f572f5f2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.335890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.336608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.337252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.338054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] PATH_INFO: `/v3/projects/63cd54835b0244feba41764e026213f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.339498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63cd54835b0244feba41764e026213f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.342470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.345513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.346145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.346429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.347032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.347732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.349377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.369117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 964/1922] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:58.369117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.374763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:58.379292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddee63-d101-429c-815c-0cba4278b639 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.385445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.386162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.386550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.386932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.387841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.389980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.418941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 965/1923] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/users => generated 300 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:58.420646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.435180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330e5f-4729-468a-9488-4b7bae6848c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.441962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.442632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.442918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.443265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.444030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.445921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.459194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 966/1924] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:58.460193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.467963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63cd54835b0244feba41764e026213f8', 'request_id': 'req-86fad838-35ef-4333-8b75-30629af01629'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:58.477078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e065b-7caa-4b5c-9a76-ca7bc629ed71 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.493946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63cd54835b0244feba41764e026213f8', 'request_id': 'req-86fad838-35ef-4333-8b75-30629af01629'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:58.507469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63cd54835b0244feba41764e026213f8', 'request_id': 'req-86fad838-35ef-4333-8b75-30629af01629'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:58.548710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.549485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.549775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.550045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.550774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.550881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] The token cache is being invalidate because project 63cd54835b0244feba41764e026213f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:17:58.551179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:58.553266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323968c-7916-4bfd-9da5-1b6e2ec1bfc6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.554557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:17:58.557213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 959/1925] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:17:58 2022] DELETE /identity/v3/projects/63cd54835b0244feba41764e026213f8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:58.567611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.569349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 967/1926] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:58.570893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.571346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.571694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.576710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.620046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:58.623981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.626310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fad838-35ef-4333-8b75-30629af01629 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.636163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.637144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.637530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.639144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] PATH_INFO: `/v3/projects/49fc892336a940538b4b3317698ca5a6/users/8c2dd82f215b474d87cbae8242963644/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.644856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 968/1927] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:58.647419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.651806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.668441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fc892336a940538b4b3317698ca5a6, user_id=8c2dd82f215b474d87cbae8242963644, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.672172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de81d-cb8c-44f1-ac21-c14fb149e0df admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.675621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73043671-5293-4d86-abf8-0f3883c0e5bc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.682232 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.683927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.684456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.685005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] PATH_INFO: `/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40/users/868416e6784f40a9ad781e6d176e2298/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.689849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 960/1928] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:58 2022] PUT /identity/v3/projects/49fc892336a940538b4b3317698ca5a6/users/8c2dd82f215b474d87cbae8242963644/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:58.691103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.692197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.692197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.710890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7ac076e4d74545a6aaac51b9ad3e40, user_id=868416e6784f40a9ad781e6d176e2298, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.714114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cbf20-2afa-42d8-993e-6c8be2b70a6c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.725217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:58.726619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.736375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 969/1929] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:58 2022] PUT /identity/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40/users/868416e6784f40a9ad781e6d176e2298/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:58.736375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.736375 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.736722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.776882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] MFA Rules not processed for user `8c2dd82f215b474d87cbae8242963644`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:58.778586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:58.782507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:58.827715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 961/1930] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:58.828917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.847472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd606448-22fa-455b-8456-9e7b4772a0c3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.855992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.857484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.857769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.858036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.858721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.860794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.870194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 970/1931] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:58.871192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.876833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 962/1932] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:58.876833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.887310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4295bf-8a41-4687-b400-953c5493339f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.887535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a6b9-c10d-4fc1-9a6e-fb1ebb2b7e45 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.892695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.892852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.893310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.893457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.893762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.893859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.894037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.894145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.894678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.894785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:17:58.895028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.896641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.896753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.915364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 971/1933] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:17:58.915364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.925345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3c145-5cb7-46b7-8cd1-530e96bdc0db admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:58.929886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 963/1934] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:58.930920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.933685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:58.934420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:58.935275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:58.935554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] PATH_INFO: `/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40/users/868416e6784f40a9ad781e6d176e2298/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:58.937603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7ac076e4d74545a6aaac51b9ad3e40, user_id=868416e6784f40a9ad781e6d176e2298, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:58.941044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:58.962805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 972/1935] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:58 2022] PUT /identity/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40/users/868416e6784f40a9ad781e6d176e2298/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:58.962805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:58.969748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:59.044961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274414f-771d-472c-9dd7-655c4facba68 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.089627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad639a73-0c2b-48fd-b36a-e16de2408890 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.095751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.096342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.096822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.096969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.098049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.100785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf8b70-bef4-4c26-9d14-6e974c4995af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.147813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 973/1936] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:59.147813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a22c6-2258-42ee-b949-392507894483 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.147813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a22c6-2258-42ee-b949-392507894483 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.148154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a22c6-2258-42ee-b949-392507894483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.172352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.173064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.173352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.173630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.175643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.178554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957a37f-76b3-496d-8f36-16368046f274 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.196869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-747a22c6-2258-42ee-b949-392507894483 None None] MFA Rules not processed for user `868416e6784f40a9ad781e6d176e2298`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:59.198155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-747a22c6-2258-42ee-b949-392507894483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:59.216997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 964/1937] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:58 2022] POST /identity/v3/projects => generated 377 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:59.219110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.219758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.220252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.258416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] MFA Rules not processed for user `9c44c63a58584625a5dd60ccd9addb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:17:59.260178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:59.295111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:59.358236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 965/1938] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:59.359569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.371894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 974/1939] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:17:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:17:59.372858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a22c6-2258-42ee-b949-392507894483 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.385287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a22c6-2258-42ee-b949-392507894483 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.530113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.530982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.531489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.531770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.532013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7b7bc-1f4d-4101-826e-e10316404056 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.532487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.534918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.553648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.556207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.557214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.557510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.560894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.562810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.581988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 975/1940] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:59 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:59.582999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.587273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:59.604957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 966/1941] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:59.605937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.609887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:17:59.617736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3518b-c471-4063-a26c-4eae85c64eda admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.639132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.639819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.640106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.640360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.648226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.651516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.670272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:59.716851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 976/1942] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:17:59 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:17:59.716851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.736921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da92de4-7a16-401d-8cfb-283f648072f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.801292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.801464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750e1d5-94cc-49b0-8e17-e139de67cd19 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.801913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.802198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.802477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.830305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.832588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.833057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.833769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.834706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.838799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.841453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:17:59.858336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 977/1943] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:17:59.859593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.874595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be453c-ca0d-45b2-967f-865a3b9e697b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:17:59.876594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.878547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.880772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:17:59.881455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:17:59.881806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:17:59.882180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] PATH_INFO: `/v3/projects/5448bd72a185490dbd776284be483922/users/6cbd138b657c4bf0b616925577317d42/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:17:59.901188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5448bd72a185490dbd776284be483922, user_id=6cbd138b657c4bf0b616925577317d42, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:17:59.904048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:17:59.909250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 967/1944] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:17:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:17:59.911810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.924163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 978/1945] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:17:59 2022] PUT /identity/v3/projects/5448bd72a185490dbd776284be483922/users/6cbd138b657c4bf0b616925577317d42/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:17:59.925234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:17:59.954339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fb0f8-4faa-4669-9e6f-85503b41eb56 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.033650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1524c8c-a8fa-406e-89ad-ba581f0ef0c5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.041101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.042157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.042609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.043045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.044437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.045784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.046499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.046953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.047437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.047766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.048261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.050246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.063603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 979/1946] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:00.064583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.079239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad691-e07c-4506-b685-eb509408f267 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.084848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.085846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.086225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.088172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 968/1947] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:17:59 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:00.089191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.092567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.093580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.095895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.118012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cea1-7158-475f-a253-4b8b8fc6b642 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.119663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 980/1948] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:00.121867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.138661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96c7a0-ce85-4247-bb4c-1c3f59d985b1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.140529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.141220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.141497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.141774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] PATH_INFO: `/v3/projects/5c2df828799c44789342b0c04bd4996a/users/7a30d5de98334ad59b012de00560cba2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.146737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.147614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.147974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.149046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.149681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.151659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.158097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2df828799c44789342b0c04bd4996a, user_id=7a30d5de98334ad59b012de00560cba2, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.160774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.166557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 981/1949] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:00.167543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.178394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 969/1950] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:00 2022] PUT /identity/v3/projects/5c2df828799c44789342b0c04bd4996a/users/7a30d5de98334ad59b012de00560cba2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:00.179190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.185994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872cef-ac11-4c60-b52f-3f1535d36de0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.221881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:00.255773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b857-12b7-449a-8729-6480c3a7851c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.258027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.259325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.259325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.259510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] PATH_INFO: `/v3/projects/5448bd72a185490dbd776284be483922/users/6cbd138b657c4bf0b616925577317d42/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.261307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.261444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5448bd72a185490dbd776284be483922, user_id=6cbd138b657c4bf0b616925577317d42, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.262301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.262301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.262488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] PATH_INFO: `/v3/users/6b7aa7d9187f42209b91d4473fc6d1a1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.263726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7aa7d9187f42209b91d4473fc6d1a1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.264383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.265657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.287910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 982/1951] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:00 2022] PUT /identity/v3/projects/5448bd72a185490dbd776284be483922/users/6cbd138b657c4bf0b616925577317d42/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:00.289402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.307584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:00.318293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7aa7d9187f42209b91d4473fc6d1a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:00.325601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7aa7d9187f42209b91d4473fc6d1a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:00.326305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7aa7d9187f42209b91d4473fc6d1a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:00.326932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7aa7d9187f42209b91d4473fc6d1a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:00.335328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3bb9ce52-75cc-4805-87cc-f9f4fc33d722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7aa7d9187f42209b91d4473fc6d1a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:00.354971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 970/1952] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:00 2022] DELETE /identity/v3/users/6b7aa7d9187f42209b91d4473fc6d1a1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:00.356248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1f419-bb9b-452b-93be-a561580ff4a1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.357126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1f419-bb9b-452b-93be-a561580ff4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.357413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1f419-bb9b-452b-93be-a561580ff4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.360320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cf89-24d6-4a05-af5c-68ccbcf2a589 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.385815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ead1f419-bb9b-452b-93be-a561580ff4a1 None None] MFA Rules not processed for user `7a30d5de98334ad59b012de00560cba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:00.387085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ead1f419-bb9b-452b-93be-a561580ff4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.427765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.428447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.429131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.429303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] PATH_INFO: `/v3/projects/8750005977fa4ef78711211912eb8582/users/fe593dd1ea6d428e8935f2d21c59cf77/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.446025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8750005977fa4ef78711211912eb8582, user_id=fe593dd1ea6d428e8935f2d21c59cf77, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.449123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36a404-f722-4bfc-8162-bf92b1411ce0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.464240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 971/1953] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:00.465505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.466336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.467037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.467329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 983/1954] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:00 2022] PUT /identity/v3/projects/8750005977fa4ef78711211912eb8582/users/fe593dd1ea6d428e8935f2d21c59cf77/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:00.472445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.472841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.473217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.497847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:00.498059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] MFA Rules not processed for user `6cbd138b657c4bf0b616925577317d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:00.499099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.499251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.585901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 972/1955] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:00.588832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.590639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.599041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:00.616383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 984/1956] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:00.616383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.616383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.645660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:00.688787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e496f30-8442-4bfa-9633-a3426a0672d1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.693844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.694580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.694863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.695137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.695816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.695933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7aea-2823-445d-950b-e24f8c3be5b5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.697444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.700866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.701488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.701797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.702126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.708764 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 973/1957] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:00.709627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.711367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.723215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e108-474b-4cd5-a44a-f8916c525a18 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.726912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.727544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.727831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.728112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.739015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.740673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.752900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 974/1958] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:00.753708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.755009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:00.765569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.767035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.779189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 985/1959] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:00.780070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.793746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e030179-b2cf-401e-8dc0-3cc0fe2ff56b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.799698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.800538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.800783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.801069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] PATH_INFO: `/v3/projects/8750005977fa4ef78711211912eb8582/users/fe593dd1ea6d428e8935f2d21c59cf77/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.803079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8750005977fa4ef78711211912eb8582, user_id=fe593dd1ea6d428e8935f2d21c59cf77, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:00.806376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:00.819337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 986/1960] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:00 2022] PUT /identity/v3/projects/8750005977fa4ef78711211912eb8582/users/fe593dd1ea6d428e8935f2d21c59cf77/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:00.824677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1e98-00ad-4a82-9fc2-381dbcdc81db service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.847401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:00.905346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.906523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.907078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.907601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.917731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba822aa7-5b52-462b-9d72-278bedfb9814 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:00.928485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:00.930403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:00.930753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:00.931609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:00.933495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:01.008956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:01.010935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:01.024696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 975/1961] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:01.027967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:01.032777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:01.034672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebaa64-5b5c-4d37-8c56-1b05284e0868 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:01.056329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 987/1962] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:01.553277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:01.555745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:01.592431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c058-dd81-4c10-928d-2726b6faf67a service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:01.607889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:01.610813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:01.612165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:01.613768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] PATH_INFO: `/v3/users/8c2dd82f215b474d87cbae8242963644` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:01.621554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2dd82f215b474d87cbae8242963644)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:01.628770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:01.668027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:01.787292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2dd82f215b474d87cbae8242963644'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:01.806187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2dd82f215b474d87cbae8242963644'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:01.819451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2dd82f215b474d87cbae8242963644'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:01.820300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2dd82f215b474d87cbae8242963644'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:01.832934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2dd82f215b474d87cbae8242963644'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:01.873138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 976/1963] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:01 2022] DELETE /identity/v3/users/8c2dd82f215b474d87cbae8242963644 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:01.878233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:01.879621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:01.880253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:01.920190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:01.921869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:01.989445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 988/1964] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:01.998999 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.000786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.063615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89cb06-2232-458d-96f0-eb29adf948c1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:02.067961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:02.068736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.069059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.069295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.079521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:02.081006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:02.092305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 977/1965] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:02.347127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.348705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.361385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db712672-01c1-4f31-af0f-fc082166b86d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:02.367897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:02.368602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.368881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.369355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] PATH_INFO: `/v3/projects/94a5609af9f54858b048ce9b7b33052e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.370504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a5609af9f54858b048ce9b7b33052e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:02.373263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:02.462327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a5609af9f54858b048ce9b7b33052e', 'request_id': 'req-373cea9f-11d6-40da-a95c-1961ce740deb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.472754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a5609af9f54858b048ce9b7b33052e', 'request_id': 'req-373cea9f-11d6-40da-a95c-1961ce740deb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.494615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a5609af9f54858b048ce9b7b33052e', 'request_id': 'req-373cea9f-11d6-40da-a95c-1961ce740deb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.515157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] The token cache is being invalidate because project 94a5609af9f54858b048ce9b7b33052e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:02.517513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.518642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.524891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.525585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 989/1966] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:02 2022] DELETE /identity/v3/projects/94a5609af9f54858b048ce9b7b33052e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:02.536402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef9f7f-b4e2-4b0c-b23d-ad50f9c1e365 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:02.596492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.599934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.659600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:02.660891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.661351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.661850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] PATH_INFO: `/v3/users/868416e6784f40a9ad781e6d176e2298` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.664883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] RBAC: Authorizing `identity:delete_user(user_id=868416e6784f40a9ad781e6d176e2298)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:02.667704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:02.715448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cea9f-11d6-40da-a95c-1961ce740deb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:02.729447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:02.732672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:02.734681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.734681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.734902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] PATH_INFO: `/v3/projects/49fc892336a940538b4b3317698ca5a6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.736239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fc892336a940538b4b3317698ca5a6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:02.738153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:02.743498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868416e6784f40a9ad781e6d176e2298'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.752909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868416e6784f40a9ad781e6d176e2298'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.753702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868416e6784f40a9ad781e6d176e2298'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.754379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868416e6784f40a9ad781e6d176e2298'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.763898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9229ee06-5306-4bfd-b92c-bee8993b813c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868416e6784f40a9ad781e6d176e2298'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.772851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:02.790197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 978/1967] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:02 2022] DELETE /identity/v3/users/868416e6784f40a9ad781e6d176e2298 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:02.791972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.792269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.792600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.805436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fc892336a940538b4b3317698ca5a6', 'request_id': 'req-7e54f082-9162-47b4-9c10-dabfe291bf74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.816255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fc892336a940538b4b3317698ca5a6', 'request_id': 'req-7e54f082-9162-47b4-9c10-dabfe291bf74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.830782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fc892336a940538b4b3317698ca5a6', 'request_id': 'req-7e54f082-9162-47b4-9c10-dabfe291bf74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.850207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] The token cache is being invalidate because project 49fc892336a940538b4b3317698ca5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:02.850601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.851665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:02.860153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 990/1968] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:02 2022] DELETE /identity/v3/projects/49fc892336a940538b4b3317698ca5a6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:02.860626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] MFA Rules not processed for user `fe593dd1ea6d428e8935f2d21c59cf77`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:02.861511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.861857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.863545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.968141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 979/1969] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:02.969674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54f082-9162-47b4-9c10-dabfe291bf74 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:02.979448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:02.982031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:02.982708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:02.983039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:02.983736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] PATH_INFO: `/v3/users/6cbd138b657c4bf0b616925577317d42` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:02.985400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:02.985855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbd138b657c4bf0b616925577317d42)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:02.988595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:03.060611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbd138b657c4bf0b616925577317d42'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.068794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbd138b657c4bf0b616925577317d42'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.069827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbd138b657c4bf0b616925577317d42'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.072108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386dcfb9-3997-4a72-8ed1-7a2f58dd7ca2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:03.078130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:03.078914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.079320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbd138b657c4bf0b616925577317d42'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.080132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.081059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.083468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.096643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbd138b657c4bf0b616925577317d42'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.106369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 991/1970] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:02 2022] DELETE /identity/v3/users/6cbd138b657c4bf0b616925577317d42 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:03.142572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:03.162319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:03.164427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:03.176530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 980/1971] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:03.611143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:03.612953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.649986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:03.652490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.657631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:03.680989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceabfc6-1947-4c9d-8c40-2096cdfae833 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:03.707843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feab8-f9b4-4c67-a8bf-96269ca6dd73 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:03.712655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:03.713437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.713751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.714031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] PATH_INFO: `/v3/projects/5448bd72a185490dbd776284be483922` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.715189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5448bd72a185490dbd776284be483922)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:03.717522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:03.746662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:03.747880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.748163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.748438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] PATH_INFO: `/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.750915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7ac076e4d74545a6aaac51b9ad3e40)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:03.753272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:03.777532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5448bd72a185490dbd776284be483922', 'request_id': 'req-9ed582e5-2336-4ce2-a8a5-56477dcf88af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.787103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5448bd72a185490dbd776284be483922', 'request_id': 'req-9ed582e5-2336-4ce2-a8a5-56477dcf88af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.797207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5448bd72a185490dbd776284be483922', 'request_id': 'req-9ed582e5-2336-4ce2-a8a5-56477dcf88af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.810406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] The token cache is being invalidate because project 5448bd72a185490dbd776284be483922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:03.810700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.811419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9ed582e5-2336-4ce2-a8a5-56477dcf88af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.813287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 992/1972] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:03 2022] DELETE /identity/v3/projects/5448bd72a185490dbd776284be483922 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:03.814728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7ac076e4d74545a6aaac51b9ad3e40', 'request_id': 'req-cb459ab1-03da-4a69-b5c1-b5e34ada5246'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.819101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aec-cb43-4bc8-99d9-c53f97954ac7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.819517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aec-cb43-4bc8-99d9-c53f97954ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.819899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aec-cb43-4bc8-99d9-c53f97954ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.822489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7ac076e4d74545a6aaac51b9ad3e40', 'request_id': 'req-cb459ab1-03da-4a69-b5c1-b5e34ada5246'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.834228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7ac076e4d74545a6aaac51b9ad3e40', 'request_id': 'req-cb459ab1-03da-4a69-b5c1-b5e34ada5246'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.844018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-107d8aec-cb43-4bc8-99d9-c53f97954ac7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:03.845308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-107d8aec-cb43-4bc8-99d9-c53f97954ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.849056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] The token cache is being invalidate because project ea7ac076e4d74545a6aaac51b9ad3e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:03.849358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.850144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cb459ab1-03da-4a69-b5c1-b5e34ada5246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:03.852743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 981/1973] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:03 2022] DELETE /identity/v3/projects/ea7ac076e4d74545a6aaac51b9ad3e40 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:03.857232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c10b-98d7-4a07-8e78-16d9f0c66197 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.857556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c10b-98d7-4a07-8e78-16d9f0c66197 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.857844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c10b-98d7-4a07-8e78-16d9f0c66197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.880704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bd59c10b-98d7-4a07-8e78-16d9f0c66197 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:03.881077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bd59c10b-98d7-4a07-8e78-16d9f0c66197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.914114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 993/1974] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:03.923251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.923555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.923834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.943743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:03.944974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.952784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 994/1975] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:03.956947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:03.957408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 982/1976] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:03.965113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.965407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.965721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.966052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09d75c-aa66-44cd-8270-a7e47169c969 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:03.970690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:03.971291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:03.971575 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:03.971850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:03.972491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:03.972792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:03.974400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:03.987147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:03.988058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 995/1977] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:03.988416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:03.992223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:03.996738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 983/1978] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:04.002259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.005882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912dc2-b747-4e47-9518-6d787f650ef8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.010746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.011096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019efe0a-40bb-4b22-b011-d4e8e22b67bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.011399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.011690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.011991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.012657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.014747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.015630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.016214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.016497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.016773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.017411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:04.017713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.019307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.031276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 984/1979] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:04.035626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.043198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.053356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 996/1980] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:03 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:04.053547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061e77-824f-498a-823d-78b04413ba7b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.054251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.076955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.129420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.130120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.130416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6e28c-9ff1-426f-bb13-a05f69e6ad63 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.130574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.130965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.132788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.134418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.137235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.137872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.138153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.138426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.139285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.145003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.171407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 985/1981] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:04.174917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.179519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 997/1982] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:04.183788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.203810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662d8e1-3828-486f-96a7-fe170f6a2c5b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.271122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106aec-b63f-4ac4-a883-37d1500e460c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.277131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.277835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.278851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.279201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.280470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.282478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.285333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.286059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.286413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.286747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.287514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.289200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.300847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 998/1983] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:04.304787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.315046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 986/1984] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:04.315538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874b85a-c6ac-463e-b2c5-0990b5a61272 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.315828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.321042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.321754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.322129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.322498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.323291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.324983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.326609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90933d5a-29d2-421f-95cb-772668d51aa3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.331742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.332346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.332635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.332905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.333550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.335290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.338183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 999/1985] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:04.342912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.353441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 987/1986] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:04.354355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.355848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed631-e8d8-42dd-b783-53d4fe8413ab admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.361225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.362178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.362476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.362941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] PATH_INFO: `/v3/projects/8444df24a9474e4baa683ae9b8a710c8/users/c7ca97d821bc4078bc1a22cbc4a909a4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.364987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93446-0f37-4547-9435-076be2a71a8a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.369797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.370477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.370836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.371180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.371956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.373635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.382455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8444df24a9474e4baa683ae9b8a710c8, user_id=c7ca97d821bc4078bc1a22cbc4a909a4, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.387260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 988/1987] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:04.387427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dabb7-0bf9-40e6-9a24-8440150f400b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.390068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.401272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113552-66af-4567-aa9f-f323676aa0b7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.413641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1000/1988] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:04 2022] PUT /identity/v3/projects/8444df24a9474e4baa683ae9b8a710c8/users/c7ca97d821bc4078bc1a22cbc4a909a4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:04.416671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.417097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.417383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.453220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] MFA Rules not processed for user `c7ca97d821bc4078bc1a22cbc4a909a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:04.454557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:04.457502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.466771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.467943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.468397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.468825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] PATH_INFO: `/v3/projects/629d7ac761af4eaab00a8d433f0bd45e/users/02cd9dacb2db4643ae20b407a7ab0ae3/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.484889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629d7ac761af4eaab00a8d433f0bd45e, user_id=02cd9dacb2db4643ae20b407a7ab0ae3, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.487579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9e7fd-f384-4e82-ae12-7fe1ce34b250 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.501601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 989/1989] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:04 2022] PUT /identity/v3/projects/629d7ac761af4eaab00a8d433f0bd45e/users/02cd9dacb2db4643ae20b407a7ab0ae3/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:04.512560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.512850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.513132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.516011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.527744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1001/1990] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:04.532756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.542629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] MFA Rules not processed for user `02cd9dacb2db4643ae20b407a7ab0ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:04.543582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023c621-e434-45cc-bf85-e1c8df4e10d9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.544452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:04.605826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.606888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.607180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.607503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.608150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.610160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.624100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 990/1991] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:04.639743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.646047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:04.655202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1002/1992] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:04.656311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.734936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7d7f-c39d-41f8-9523-4ac603833ed8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.741439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.742074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.742358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.742637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.742777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635fabe-11ce-4491-9fde-2e724a9b27d3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.743349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.745378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.773207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1003/1993] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:04 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:04.776986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.787292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f6b6-fce3-4972-8a05-1962cb77138f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.792646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.793398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.794417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.795020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.795963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.797688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.805528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.806282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.806627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.806911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.810233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1004/1994] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:04.816619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.827613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d60c2-7922-4830-b784-8aa7f54fbbf7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.829651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.833979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.834715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.834925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.835198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] PATH_INFO: `/v3/projects/7cffffe01bd3457d95c598b88b6115a8/users/0ceb0f1c1f97469bb58d20038e9bc1bd/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.850515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cffffe01bd3457d95c598b88b6115a8, user_id=0ceb0f1c1f97469bb58d20038e9bc1bd, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.853211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.866724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1005/1995] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:04 2022] PUT /identity/v3/projects/7cffffe01bd3457d95c598b88b6115a8/users/0ceb0f1c1f97469bb58d20038e9bc1bd/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:04.870346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.873422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.874865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.884557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 991/1996] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:04.913100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:04.950831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42566761-8b8f-4b92-ab4b-d8d9d9facc22 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:04.959721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:04.960342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:04.960622 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:04.960896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:04.961550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:04.963182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956978b-9acd-472c-b78b-c1a288d288bb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:04.978059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1006/1997] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:04.982805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:04.996576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9808f92-b7e8-4fdb-b8e2-375f15fff7cb service glance] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:05.005505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:05.006316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:05.006789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:05.007236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] PATH_INFO: `/v3/projects/7cffffe01bd3457d95c598b88b6115a8/users/0ceb0f1c1f97469bb58d20038e9bc1bd/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:05.009523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cffffe01bd3457d95c598b88b6115a8, user_id=0ceb0f1c1f97469bb58d20038e9bc1bd, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:05.012475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:05.026034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 992/1998] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:04 2022] PUT /identity/v3/projects/7cffffe01bd3457d95c598b88b6115a8/users/0ceb0f1c1f97469bb58d20038e9bc1bd/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:05.034744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:05.035196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:05.035410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:05.068924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] MFA Rules not processed for user `0ceb0f1c1f97469bb58d20038e9bc1bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:05.070142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:05.145809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1007/1999] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:05.156774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:05.158578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:05.222575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906308a-852c-4832-93c3-35a57be793c4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:05.226453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:05.227038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:05.227353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:05.227635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:05.297415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:05.299954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:05.310934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 993/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:07.864523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:07.867987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:07.882825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f924a-be74-4a40-a630-cca7ea0f694c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:07.886701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:07.887350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:07.887667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:07.887971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:07.891075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:07.988093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:07.989734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:08.004242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1008/2001] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:15.305820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:17.276214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:17.285275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef96e58-194b-4f6d-a7a6-15786b3160d7 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:17.289109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:17.289733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:17.290014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:17.290288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:17.301035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:17.302692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874f8f-ff9a-4f7a-a010-d5df90fe090d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:17.316712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 994/2002] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:17.998743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=77 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:27.308911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:29.099483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:29.164123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b2246-4c03-4719-81a3-68e95c09fc55 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:29.168949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:29.169624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:29.169914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:29.170194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] PATH_INFO: `/v3/users/c7ca97d821bc4078bc1a22cbc4a909a4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:29.171467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ca97d821bc4078bc1a22cbc4a909a4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:29.173508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:29.238601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ca97d821bc4078bc1a22cbc4a909a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.251754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ca97d821bc4078bc1a22cbc4a909a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.252519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ca97d821bc4078bc1a22cbc4a909a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.263512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ca97d821bc4078bc1a22cbc4a909a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.276856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ca97d821bc4078bc1a22cbc4a909a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.288028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1009/2003] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:29 2022] DELETE /identity/v3/users/c7ca97d821bc4078bc1a22cbc4a909a4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:29.289859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:29.290318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:29.292135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:29.313268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:29.314474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:29.378912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 995/2004] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:29.388516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:29.390120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:29.454412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ac9f-6acd-45c2-a9b6-1f165d5197f6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:29.458432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:29.459108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:29.459498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:29.459750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:29.485479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:29.487138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:29.501747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1010/2005] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:29.786377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:29.788098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:29.809065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765f358-5de7-44ef-8da5-1773b7f6d367 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:29.817901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:29.818851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:29.819205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:29.819566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] PATH_INFO: `/v3/projects/8444df24a9474e4baa683ae9b8a710c8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:29.821022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8444df24a9474e4baa683ae9b8a710c8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:29.823497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:29.874089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8444df24a9474e4baa683ae9b8a710c8', 'request_id': 'req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.880409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8444df24a9474e4baa683ae9b8a710c8', 'request_id': 'req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.886179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8444df24a9474e4baa683ae9b8a710c8', 'request_id': 'req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.899545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] The token cache is being invalidate because project 8444df24a9474e4baa683ae9b8a710c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:29.899839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.900481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:29.901983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 996/2006] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:29 2022] DELETE /identity/v3/projects/8444df24a9474e4baa683ae9b8a710c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:29.906827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:29.916705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b03e-849d-4fc9-9bfe-bb75117e4b34 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:29.934906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:29.978844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:29.979520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:29.979829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:29.980104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] PATH_INFO: `/v3/users/0ceb0f1c1f97469bb58d20038e9bc1bd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:29.981377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ceb0f1c1f97469bb58d20038e9bc1bd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:29.983654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.032274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ceb0f1c1f97469bb58d20038e9bc1bd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.039111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ceb0f1c1f97469bb58d20038e9bc1bd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.039806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ceb0f1c1f97469bb58d20038e9bc1bd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.047433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ceb0f1c1f97469bb58d20038e9bc1bd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.060679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a6f1d877-ca3f-48a9-b519-71a8b07d7809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ceb0f1c1f97469bb58d20038e9bc1bd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.066684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1011/2007] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:29 2022] DELETE /identity/v3/users/0ceb0f1c1f97469bb58d20038e9bc1bd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:30.353282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.356160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:30.369846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772506-4da1-4ab6-9fd4-ccdb1e60969a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.430750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.431388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.431672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.431980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] PATH_INFO: `/v3/projects/7cffffe01bd3457d95c598b88b6115a8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.433162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cffffe01bd3457d95c598b88b6115a8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.435819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.467741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:30.488550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cffffe01bd3457d95c598b88b6115a8', 'request_id': 'req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.495298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cffffe01bd3457d95c598b88b6115a8', 'request_id': 'req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.502450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cffffe01bd3457d95c598b88b6115a8', 'request_id': 'req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.522461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] The token cache is being invalidate because project 7cffffe01bd3457d95c598b88b6115a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:30.522461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.523395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14d879d6-df8e-4fe8-be0f-4e6a2ae37ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:30.526754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 997/2008] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:30 2022] DELETE /identity/v3/projects/7cffffe01bd3457d95c598b88b6115a8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:30.531690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.532191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.532394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.556090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:30.557695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:30.631457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1012/2009] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:30.643272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.643272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.643272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.662614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:30.663832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:30.672573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 998/2010] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:30.676925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.689481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cd198-cb8a-4ec2-85a5-2efa958e9e28 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.695787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.696432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.696767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.697051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.697702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:30.698041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.699874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.714534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1013/2011] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:18:30.718381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.732164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69efeaa-fd87-487d-8578-032fad34b730 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.738499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.740230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.741003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.741286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.741974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.745001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.782389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 999/2012] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:30 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:30.783128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.796320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b938c-1895-4ec5-bd1d-63183a46e8d3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.823330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.825916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:30.835693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:30.889847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.890632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.890926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.891203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.892135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.895266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.932919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8e129-2acf-41dd-b664-16de112febe0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.943535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1014/2013] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:30 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:30.945125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.946688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.947478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.947770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.948050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] PATH_INFO: `/v3/users/7a30d5de98334ad59b012de00560cba2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.949449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a30d5de98334ad59b012de00560cba2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.952007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.957703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701abfe6-4b1c-442f-8d34-de9eb73daa17 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.962463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.963072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.963402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.963687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:30.964337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:30.966431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:30.977935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1015/2014] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:30.982724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:30.993494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15605-aa0c-4157-a8ec-dbec08e84ed7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:30.998449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:30.999050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:30.999368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:30.999646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.000288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.002151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.007425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a30d5de98334ad59b012de00560cba2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.013120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1016/2015] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:31.014554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a30d5de98334ad59b012de00560cba2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.015318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a30d5de98334ad59b012de00560cba2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.015971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a30d5de98334ad59b012de00560cba2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.017475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.019590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.024677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d5795c4-564d-4fad-b338-bd6fcd6416c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a30d5de98334ad59b012de00560cba2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.033372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf0a29-2744-4086-8a60-4f31baf12d63 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.038015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.038815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.039100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.039407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] PATH_INFO: `/v3/projects/6602823648eb4b12b5ad97cab505b13d/users/8e2f27dbca9a4467bf5287ae7c8b89c8/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.044658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1000/2016] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:30 2022] DELETE /identity/v3/users/7a30d5de98334ad59b012de00560cba2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:31.048564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.049099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.049331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.057640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6602823648eb4b12b5ad97cab505b13d, user_id=8e2f27dbca9a4467bf5287ae7c8b89c8, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.058116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:31.060858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fda32-4108-4f53-a904-8a9daae06344 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.073031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:31.075067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.076988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1017/2017] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:31 2022] PUT /identity/v3/projects/6602823648eb4b12b5ad97cab505b13d/users/8e2f27dbca9a4467bf5287ae7c8b89c8/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:31.088443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.089016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.089016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.117145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] MFA Rules not processed for user `8e2f27dbca9a4467bf5287ae7c8b89c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:31.118354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.137552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1001/2018] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:31.148560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.150435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.185241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1018/2019] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:31.190259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.202447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c17569-c3a1-4cdd-a304-1d5adc423ce4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.207840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.208449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.208732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.209007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.209662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.211632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.212126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c6947-1b60-4d3c-8454-9d13f7ef192c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.215912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.216544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.216813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.217102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.226425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.227900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.240085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1002/2020] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:31.240765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1019/2021] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:31.244698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.255413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda47baa-bed4-4b0e-8312-2ff5b5619b8c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.314747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.315388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.315674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.315955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.316603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.318621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.345846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1003/2022] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:31.347179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.360011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f1ac5-223a-4179-9fd9-0eb6c6dd2fc1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.364793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.365436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.365719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.365996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.366677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.368324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.382070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1020/2023] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:31.389866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.400200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a5a99-897b-4c98-9a74-c0315b9767fb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.405523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.406133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.406574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.406857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.407527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.409154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee1f36-6211-48a2-8654-08d7ce5233f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.420458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1004/2024] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:31.424463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.435409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7778be-631c-437f-a927-e815372990b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.440943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.441644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.441953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.442235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] PATH_INFO: `/v3/projects/8b6c8e6074194b42b65a43d6fb10fca5/users/efa6850242c94f27a5ce5fecd438c675/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.458032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6c8e6074194b42b65a43d6fb10fca5, user_id=efa6850242c94f27a5ce5fecd438c675, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.460754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.476064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1021/2025] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:31 2022] PUT /identity/v3/projects/8b6c8e6074194b42b65a43d6fb10fca5/users/efa6850242c94f27a5ce5fecd438c675/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:31.489522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.489917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.490416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.526558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.526880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] MFA Rules not processed for user `efa6850242c94f27a5ce5fecd438c675`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:31.528839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.529680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.536568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:31.550300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:31.616311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a04f7-6449-4329-9c0d-c6c6b434b090 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.621919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.623126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.623632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.624061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] PATH_INFO: `/v3/projects/5c2df828799c44789342b0c04bd4996a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.627246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2df828799c44789342b0c04bd4996a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.630038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.636730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1005/2026] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:31.640026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.654345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469fb54-025b-46f3-a9e0-40da40646ae9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.660571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.661223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.661661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.661945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.662570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.664826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.698031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1006/2027] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:31.702531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.716021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2df828799c44789342b0c04bd4996a', 'request_id': 'req-e019125b-279f-4ae3-b1b3-4c4a750a17af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.724690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2df828799c44789342b0c04bd4996a', 'request_id': 'req-e019125b-279f-4ae3-b1b3-4c4a750a17af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.731653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2df828799c44789342b0c04bd4996a', 'request_id': 'req-e019125b-279f-4ae3-b1b3-4c4a750a17af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.742117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] The token cache is being invalidate because project 5c2df828799c44789342b0c04bd4996a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:31.742415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.743061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e019125b-279f-4ae3-b1b3-4c4a750a17af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:31.744593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1022/2028] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:31 2022] DELETE /identity/v3/projects/5c2df828799c44789342b0c04bd4996a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:31.750843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4986f4-ecf3-4e08-9003-6b0dc9b26cf2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.751129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4986f4-ecf3-4e08-9003-6b0dc9b26cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.751436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4986f4-ecf3-4e08-9003-6b0dc9b26cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.771068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202a4c1-5ded-40b3-a6f8-da7ba14a5cd4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.771505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e4986f4-ecf3-4e08-9003-6b0dc9b26cf2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:31.772820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e4986f4-ecf3-4e08-9003-6b0dc9b26cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.829073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.829692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.829976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.830261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.830899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.832292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1023/2029] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:31.833057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.839746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.840074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.840343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.855538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1007/2030] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:31.859824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.860074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:31.861265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.861679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:31.869788 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1024/2031] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:31.873737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.874742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe449c-1157-4645-9b28-f87e965364f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.881113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.881756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.882046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.882331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.882987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.883605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2c9c2-4964-47a1-84e5-10df6dc21658 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.884818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.889040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.889644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.889920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.890191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.890843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:31.891144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.892909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.896704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1008/2032] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:31.900895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.910852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1025/2033] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:31.912016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:31.922679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b72fee-26de-4fa2-b944-b3974a55f0b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.922982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d786f-c150-4930-a5b5-cfc20573770b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:31.928968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.929574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.929860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.930195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.930822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.932988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.933934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:31.934996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:31.935528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:31.936056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] PATH_INFO: `/v3/projects/dfcb40889a4b4b888a48b97548fffb09/users/90982b112de04e76830b46f5e5e2479f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:31.954358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:31.964689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcb40889a4b4b888a48b97548fffb09, user_id=90982b112de04e76830b46f5e5e2479f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:31.968855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:31.975801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:31.977577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1026/2034] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:31.982306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.006330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1009/2035] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:31 2022] PUT /identity/v3/projects/dfcb40889a4b4b888a48b97548fffb09/users/90982b112de04e76830b46f5e5e2479f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:32.007458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.063917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b53dc-96a2-4a5d-abc7-1126ec425547 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.084848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a87d77-136b-48c4-b5ef-cd47e39965ca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.094471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.095516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.095790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.096065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.097038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.099663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.115718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1010/2036] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:32.116597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.126760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2cec-7583-4591-84e7-ced785dc8641 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.130740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.131398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.131684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.131959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.132082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.132740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.132920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.133098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.133392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] PATH_INFO: `/v3/projects/dfcb40889a4b4b888a48b97548fffb09/users/90982b112de04e76830b46f5e5e2479f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.135407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.135525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcb40889a4b4b888a48b97548fffb09, user_id=90982b112de04e76830b46f5e5e2479f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.138575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c7641-8143-4ac6-965f-986bc710a123 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.153612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1011/2037] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:32 2022] PUT /identity/v3/projects/dfcb40889a4b4b888a48b97548fffb09/users/90982b112de04e76830b46f5e5e2479f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:32.163304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1027/2038] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:31 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:32.165288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578995-9927-490b-b7cd-116517582c4e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.165584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578995-9927-490b-b7cd-116517582c4e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.165874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8578995-9927-490b-b7cd-116517582c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.172220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.182246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7d5cd-dc86-4fd0-86d9-1a96d94d2ea2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.196619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d8578995-9927-490b-b7cd-116517582c4e None None] MFA Rules not processed for user `90982b112de04e76830b46f5e5e2479f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:32.198049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8578995-9927-490b-b7cd-116517582c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:32.247772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.248836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.249321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.249747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.250817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.252794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.267849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1028/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:32.269038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.275141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1012/2040] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:32.282439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c9e6-671c-4e7c-aed6-638409984bd7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.288282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8578995-9927-490b-b7cd-116517582c4e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.289763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8578995-9927-490b-b7cd-116517582c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:32.291405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.292226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.292516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.292799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.294034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.295713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.317747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1029/2041] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:32.318889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.329211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978a9c3-5cd5-46a7-acd7-361231e71dec admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.334484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.335138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.335496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.335698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] PATH_INFO: `/v3/projects/a9bdb6983382496aa4d0363bec53e29d/users/ce579ebd16f04855aca97038af95ec7f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.350089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:32.354169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bdb6983382496aa4d0363bec53e29d, user_id=ce579ebd16f04855aca97038af95ec7f, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.362223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164172b-ff0a-4d9b-b635-5124f60b0aa3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.369877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8578995-9927-490b-b7cd-116517582c4e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.378152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1030/2042] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:32 2022] PUT /identity/v3/projects/a9bdb6983382496aa4d0363bec53e29d/users/ce579ebd16f04855aca97038af95ec7f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:32.390180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.390465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.390742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.391402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:32.427367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] MFA Rules not processed for user `ce579ebd16f04855aca97038af95ec7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:32.428732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:32.434972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.435689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.435987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.436183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.438178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:32.515586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.517138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.524605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1031/2043] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:32.531325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1013/2044] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:32.563697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:32.565195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:32.577661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f5188-f897-4e2a-bce6-81a187723982 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:32.581659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:32.582023 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:32.582307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:32.582571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:32.592827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:32.594319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:32.607712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1032/2045] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:35.252669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:35.254245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:35.328277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48000-0dc7-4384-9967-d9b418486c58 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:35.335431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:35.336121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:35.336415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:35.336702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] PATH_INFO: `/v3/users/9c44c63a58584625a5dd60ccd9addb95` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:35.337934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c44c63a58584625a5dd60ccd9addb95)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:35.339881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:35.368488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:35.379115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015abd5-fa00-4f01-b90a-7b2917f8ed35 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:35.383303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:35.383930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:35.384272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:35.384550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:35.386688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:35.404700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c44c63a58584625a5dd60ccd9addb95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.418464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c44c63a58584625a5dd60ccd9addb95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.419212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c44c63a58584625a5dd60ccd9addb95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.419940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c44c63a58584625a5dd60ccd9addb95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.434548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c44c63a58584625a5dd60ccd9addb95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.447257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:35.459721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1014/2046] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:35 2022] DELETE /identity/v3/users/9c44c63a58584625a5dd60ccd9addb95 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:35.472361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:35.479333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:35.484482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1033/2047] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:35.658803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:35.660853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:35.757832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902738a7-d13e-40b4-a34c-70957293365d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:35.764556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:35.765198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:35.765865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:35.766141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] PATH_INFO: `/v3/projects/06c33fcecae44f58a68b42a87984925f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:35.767766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c33fcecae44f58a68b42a87984925f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:35.770344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:35.821481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c33fcecae44f58a68b42a87984925f', 'request_id': 'req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.828675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c33fcecae44f58a68b42a87984925f', 'request_id': 'req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.835522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c33fcecae44f58a68b42a87984925f', 'request_id': 'req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.852449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] The token cache is being invalidate because project 06c33fcecae44f58a68b42a87984925f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:35.852746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.853421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:35.854941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1015/2048] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:35 2022] DELETE /identity/v3/projects/06c33fcecae44f58a68b42a87984925f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:35.859327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:35.872316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5464ea-f697-4ce2-b7a4-0986cb0fad32 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:35.941798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:35.942433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:35.942712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:35.942983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] PATH_INFO: `/v3/users/fe593dd1ea6d428e8935f2d21c59cf77` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:35.944179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe593dd1ea6d428e8935f2d21c59cf77)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:35.946133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:35.953820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:36.001115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe593dd1ea6d428e8935f2d21c59cf77'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.008407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe593dd1ea6d428e8935f2d21c59cf77'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.009097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe593dd1ea6d428e8935f2d21c59cf77'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.017347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe593dd1ea6d428e8935f2d21c59cf77'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.030127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe593dd1ea6d428e8935f2d21c59cf77'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.036013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1034/2049] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:35 2022] DELETE /identity/v3/users/fe593dd1ea6d428e8935f2d21c59cf77 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:36.104082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.106190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.165627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8176-e12b-4a65-86ab-3a06ea6ca7aa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.212612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:36.227165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.230135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.244066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.245156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.246160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.246456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.248949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.326681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.327991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bebcd5-89d2-4d08-a638-8b391f1242af admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.328286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13635f1-7b9f-4471-b952-1ece81d75e30 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.332654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.333601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.333601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.333873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] PATH_INFO: `/v3/projects/8750005977fa4ef78711211912eb8582` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.334982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8750005977fa4ef78711211912eb8582)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.337220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.340464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1016/2050] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:36.391334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8750005977fa4ef78711211912eb8582', 'request_id': 'req-23cce5bb-32c9-49f7-b43a-84e8d84df100'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.411683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8750005977fa4ef78711211912eb8582', 'request_id': 'req-23cce5bb-32c9-49f7-b43a-84e8d84df100'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.423875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8750005977fa4ef78711211912eb8582', 'request_id': 'req-23cce5bb-32c9-49f7-b43a-84e8d84df100'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.439254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] The token cache is being invalidate because project 8750005977fa4ef78711211912eb8582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:36.439254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.439697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23cce5bb-32c9-49f7-b43a-84e8d84df100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:36.441798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1035/2051] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:36 2022] DELETE /identity/v3/projects/8750005977fa4ef78711211912eb8582 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:36.448023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.448460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.448742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.474253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:36.475511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.564541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1017/2052] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:36.565962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.566319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.566668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.592760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:36.594066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:36.603459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1036/2053] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:36.608400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.622501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781e771-a702-4a7d-b095-c99557cb5020 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.630901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.631832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.632335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.633043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.634526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:36.634991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.638489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.656452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1018/2054] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:36.660720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.673491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9c95d-d2d4-4848-a4e2-7a15fcb90fe4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.680160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.681120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.681548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.681962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.682815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.685116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.690944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:36.723163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1037/2055] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:36 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:36.726816 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.737242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382657-de04-43be-8ad6-962ed828ba8f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.808160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.809452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.809802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.810009 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.811076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.812639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.846122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1019/2056] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:36 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:36.847365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.867126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3ed3b-b4ab-4e3c-a90e-07717485e8ec admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.875855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.876480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.876781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.877054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.877706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.879369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.890973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1038/2057] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:36.895079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.906735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd374-1dc2-4e78-a66a-8eaaa7f64827 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.912491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.913258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.913553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.913839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.915188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.916596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314086d0-8534-4cba-afbb-2271f4f4cf2d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.917902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:36.934389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1020/2058] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:36.935305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:36.946027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf89b81-a26d-4deb-8eec-02cd7916a21e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:36.951529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:36.952254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.952633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.952950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] PATH_INFO: `/v3/projects/0d2364d6d3c0423db3c992c22e9cd10b/users/8a1ebb195f34459da3187e2a31403019/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:36.968473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2364d6d3c0423db3c992c22e9cd10b, user_id=8a1ebb195f34459da3187e2a31403019, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:36.971144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:36.987051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1039/2059] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:36 2022] PUT /identity/v3/projects/0d2364d6d3c0423db3c992c22e9cd10b/users/8a1ebb195f34459da3187e2a31403019/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:36.998624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:36.998911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:36.999192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:37.030914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] MFA Rules not processed for user `8a1ebb195f34459da3187e2a31403019`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:37.032314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:37.100292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1021/2060] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:37.112459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:37.114367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:37.174490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a2b5c-29b7-47f2-9439-aeb34c841822 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:37.178435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:37.179043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:37.179369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:37.179647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:37.188248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:37.189769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72de1dc-9ff5-4615-91e6-917aad94fe83 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:37.200566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1040/2061] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:37.219851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:37.283941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbc101-3d32-4ca9-bb46-c3a2b5023e2c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:37.288627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:37.289262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:37.289549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:37.289830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] PATH_INFO: `/v3/users/8a1ebb195f34459da3187e2a31403019` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:37.290984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1ebb195f34459da3187e2a31403019)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:37.292962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:37.343721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1ebb195f34459da3187e2a31403019'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:37.351721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1ebb195f34459da3187e2a31403019'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:37.352453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1ebb195f34459da3187e2a31403019'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:37.353247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1ebb195f34459da3187e2a31403019'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:37.361967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1ebb195f34459da3187e2a31403019'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:37.382860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1022/2062] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:37 2022] DELETE /identity/v3/users/8a1ebb195f34459da3187e2a31403019 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:37.387850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:37.388169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:37.388453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:37.410220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:37.411462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:37.490525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1041/2063] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:37.503666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:37.505305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:37.513149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:37.622340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddaf93-b684-4582-b7c4-8d43f50d8873 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:37.633244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:37.633884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:37.634168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:37.634470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:37.650887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:37.653098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f64fc0-48cb-4cd0-bdb8-53d8d89672c9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:37.664049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1023/2064] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:38.146511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.148712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:38.166804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ede312-3777-418e-a6f0-2c06943768ab None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.173436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.174202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.174765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.175070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] PATH_INFO: `/v3/projects/0d2364d6d3c0423db3c992c22e9cd10b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.177015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2364d6d3c0423db3c992c22e9cd10b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.180050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.244216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2364d6d3c0423db3c992c22e9cd10b', 'request_id': 'req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:38.246056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:38.253406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2364d6d3c0423db3c992c22e9cd10b', 'request_id': 'req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:38.262657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2364d6d3c0423db3c992c22e9cd10b', 'request_id': 'req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:38.274845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] The token cache is being invalidate because project 0d2364d6d3c0423db3c992c22e9cd10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:38.275151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:38.275883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aafb0a62-3bc5-4f71-b59d-a51158d4ad5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:38.277420 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1042/2065] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:38 2022] DELETE /identity/v3/projects/0d2364d6d3c0423db3c992c22e9cd10b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:38.283166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.283579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.283969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.310960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:38.312651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:38.386236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1024/2066] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:38.394530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.394819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.395101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.415033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:38.416329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:38.425156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1043/2067] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:38.429545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.440178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce7259-267b-46d4-ac77-5c8ff52e7cb1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.445002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.445575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.445860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.446181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.446783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:38.447085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.448718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.460583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1025/2068] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:18:38.465586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.475130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88e640-1151-4637-a9a8-307bee4e3c8f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.480033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.481268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.481566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.481827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.482461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.484406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.515166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1044/2069] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:38 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:38.516476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.526777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d017d-2363-4d08-9c3d-638ecf04d04c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.587992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.588605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.588886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.589161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.589838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.592509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.613842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1026/2070] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:38 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:38.618252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.629052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aab35f-ca13-466e-9540-66abf659602f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.633651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.634246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.634525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.634795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.635471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.637612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.649568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1045/2071] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:38.654806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.665471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41a78-0aca-49ad-8a1d-a647b2219e12 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.671920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.672319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.672779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.673267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.674390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.676746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629435c5-da14-40b8-a1a4-8e64b45e7fd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.688434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1027/2072] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:38.692644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.703664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd46d6d-2f54-47a5-b7fa-14c0d6aa252f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.708578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.709227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.709510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.710020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] PATH_INFO: `/v3/projects/d622b32100ab44de8fad1e0beb754a0e/users/2898acc09279411a8318a6ac9505f1f9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.729323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d622b32100ab44de8fad1e0beb754a0e, user_id=2898acc09279411a8318a6ac9505f1f9, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.733213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:38.750904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1046/2073] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:38 2022] PUT /identity/v3/projects/d622b32100ab44de8fad1e0beb754a0e/users/2898acc09279411a8318a6ac9505f1f9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:38.764221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.764566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.764851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.793236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:38.797506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] MFA Rules not processed for user `2898acc09279411a8318a6ac9505f1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:38.799914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:38.874252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1028/2074] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:38.887899 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:38.889773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:38.960195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305af2a5-9ec4-41aa-8246-837b3c73c02c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:38.965519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:38.965519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:38.965962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:38.966310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:38.979531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:38.982656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42102ff-e601-4b2f-866f-271444d9d34e service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:39.009968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1047/2075] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:40.757529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:40.759190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:40.838941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0cebd-bfcf-412e-9045-2e8b4e54f90e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:40.844047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:40.844993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:40.845157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:40.845542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] PATH_INFO: `/v3/users/8e2f27dbca9a4467bf5287ae7c8b89c8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:40.847487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e2f27dbca9a4467bf5287ae7c8b89c8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:40.849558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:40.918566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e2f27dbca9a4467bf5287ae7c8b89c8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:40.927067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e2f27dbca9a4467bf5287ae7c8b89c8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:40.927873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e2f27dbca9a4467bf5287ae7c8b89c8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:40.928602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e2f27dbca9a4467bf5287ae7c8b89c8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:40.940654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e2f27dbca9a4467bf5287ae7c8b89c8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:40.966282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1029/2076] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:40 2022] DELETE /identity/v3/users/8e2f27dbca9a4467bf5287ae7c8b89c8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:40.971904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:40.972424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:40.972505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:40.981974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:40.984406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.001522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:41.003200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.025325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:41.078997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf608359-8d87-4727-bf69-1e9d02dca60c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:41.085310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:41.086061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.086321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.086674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] PATH_INFO: `/v3/users/02cd9dacb2db4643ae20b407a7ab0ae3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.088334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cd9dacb2db4643ae20b407a7ab0ae3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:41.092506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:41.126604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1048/2077] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:41.136047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:41.139674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.164832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:41.177126 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cd9dacb2db4643ae20b407a7ab0ae3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:41.207520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cd9dacb2db4643ae20b407a7ab0ae3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:41.208381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cd9dacb2db4643ae20b407a7ab0ae3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:41.209954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cd9dacb2db4643ae20b407a7ab0ae3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:41.224329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6a411246-48fc-4847-9bef-e5893a76cda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cd9dacb2db4643ae20b407a7ab0ae3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:41.252323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aeed85-8bee-466c-aae7-e7bfa6da93a3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:41.257172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:41.258514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.259010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.259796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.264968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.265214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1030/2078] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:40 2022] DELETE /identity/v3/users/02cd9dacb2db4643ae20b407a7ab0ae3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:41.269939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.270704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.270704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.286446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:41.289085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:41.296881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:41.298245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.303184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1049/2079] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:41.407350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1031/2080] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:41.420712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:41.422276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.452288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ed770-5f69-49f9-9c26-2823a0114faa service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:41.517720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:41.518488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.518831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.519179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.533438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:41.535406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:41.548761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1050/2081] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:41.881891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:41.892155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.916864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f57e0-71fd-4d42-adf9-1c3715c376f8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:41.933919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:41.934854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.935246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.935732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] PATH_INFO: `/v3/projects/629d7ac761af4eaab00a8d433f0bd45e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.937032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=629d7ac761af4eaab00a8d433f0bd45e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:41.940314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:41.955030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:41.956209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:41.975157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64399dd7-e6cb-44b9-b3e0-37398598435d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:41.985236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:41.986296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:41.986851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:41.987407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] PATH_INFO: `/v3/projects/6602823648eb4b12b5ad97cab505b13d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:41.988955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6602823648eb4b12b5ad97cab505b13d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:41.991667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.002346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:42.020446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629d7ac761af4eaab00a8d433f0bd45e', 'request_id': 'req-a25c97cc-a054-41a6-b18b-a5d04df04f4e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.028694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629d7ac761af4eaab00a8d433f0bd45e', 'request_id': 'req-a25c97cc-a054-41a6-b18b-a5d04df04f4e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.036252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629d7ac761af4eaab00a8d433f0bd45e', 'request_id': 'req-a25c97cc-a054-41a6-b18b-a5d04df04f4e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.055390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6602823648eb4b12b5ad97cab505b13d', 'request_id': 'req-14a834a8-a39c-4733-b8ed-5457942387a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.055940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] The token cache is being invalidate because project 629d7ac761af4eaab00a8d433f0bd45e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:42.056242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.057036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a25c97cc-a054-41a6-b18b-a5d04df04f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.059689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1032/2082] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:41 2022] DELETE /identity/v3/projects/629d7ac761af4eaab00a8d433f0bd45e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:42.065238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e2079-c38c-4b2e-a49e-34b592eb0660 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.065597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e2079-c38c-4b2e-a49e-34b592eb0660 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.065854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e2079-c38c-4b2e-a49e-34b592eb0660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.069889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6602823648eb4b12b5ad97cab505b13d', 'request_id': 'req-14a834a8-a39c-4733-b8ed-5457942387a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.079436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6602823648eb4b12b5ad97cab505b13d', 'request_id': 'req-14a834a8-a39c-4733-b8ed-5457942387a1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.090056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cc1e2079-c38c-4b2e-a49e-34b592eb0660 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:42.091346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc1e2079-c38c-4b2e-a49e-34b592eb0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.095206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] The token cache is being invalidate because project 6602823648eb4b12b5ad97cab505b13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:42.096384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.097635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.100321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1051/2083] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:41 2022] DELETE /identity/v3/projects/6602823648eb4b12b5ad97cab505b13d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:42.104781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.107173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.164948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:42.185640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a834a8-a39c-4733-b8ed-5457942387a1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.191127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.192151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.192634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.193106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] PATH_INFO: `/v3/users/efa6850242c94f27a5ce5fecd438c675` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.194540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efa6850242c94f27a5ce5fecd438c675)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.196770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.197277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1033/2084] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:42.206405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.206697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.206983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.243289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:42.244840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.256669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1034/2085] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:42.260587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.285334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecedda-5073-4864-9ec2-c8ac023b7ad3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.296261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.296887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.297228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.297513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.298174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:42.298487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.300582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.313679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efa6850242c94f27a5ce5fecd438c675'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.323842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1035/2086] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:42.325650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.329569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efa6850242c94f27a5ce5fecd438c675'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.331175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efa6850242c94f27a5ce5fecd438c675'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.343277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e156c-b5a6-4bac-a45e-e909a43f9105 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.346721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efa6850242c94f27a5ce5fecd438c675'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.353503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.356217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.361320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.362527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.363208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.366409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.369768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efa6850242c94f27a5ce5fecd438c675'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.378717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1052/2087] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:42 2022] DELETE /identity/v3/users/efa6850242c94f27a5ce5fecd438c675 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:42.429420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1036/2088] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:42.434952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.437230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.445847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.450697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.563955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08011bab-ca55-4dbc-b6f4-2fcc9247f145 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.569380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.570010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.570327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.570682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.571415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.573966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.580812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cd279-18cb-4d99-a77b-751a94a1c615 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.590940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.592208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.592554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.592972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] PATH_INFO: `/v3/users/2898acc09279411a8318a6ac9505f1f9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.595211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] RBAC: Authorizing `identity:delete_user(user_id=2898acc09279411a8318a6ac9505f1f9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.600985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.607357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1053/2089] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:42.610720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.632562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7505-fd63-4f85-9f63-77beae4a19c0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.639784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.640549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.640709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.647143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.647522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.650115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.666770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:42.682512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1054/2090] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:42.683859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.703848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1d40-92d2-41f8-b1f8-f129bb9d3202 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.712768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.713525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.713872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.714496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.716351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.719635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.725742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2898acc09279411a8318a6ac9505f1f9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.744055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1055/2091] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:42.746750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2898acc09279411a8318a6ac9505f1f9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.747567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2898acc09279411a8318a6ac9505f1f9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.749630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2898acc09279411a8318a6ac9505f1f9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.749886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.753682 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.760955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2898acc09279411a8318a6ac9505f1f9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:42.774951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cc0bf-b91d-4fbe-b432-02bf8a7374c3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.781443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:42.782749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.782749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.782993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] PATH_INFO: `/v3/projects/0e1f4336492e410899d7c9f53af824fb/users/baba9ef2b1234fb8a3fed5ae472f8195/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.793761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1037/2092] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:42 2022] DELETE /identity/v3/users/2898acc09279411a8318a6ac9505f1f9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:42.794794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:42.796727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.823640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1f4336492e410899d7c9f53af824fb, user_id=baba9ef2b1234fb8a3fed5ae472f8195, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:42.829209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec650ea6-bb6d-451f-9de0-6c76daae41ac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:42.858422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1056/2093] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:42 2022] PUT /identity/v3/projects/0e1f4336492e410899d7c9f53af824fb/users/baba9ef2b1234fb8a3fed5ae472f8195/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:42.864218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999734c-bb9e-4323-a069-fa04d7086cf7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.866273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999734c-bb9e-4323-a069-fa04d7086cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.867208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999734c-bb9e-4323-a069-fa04d7086cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:42.896192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8999734c-bb9e-4323-a069-fa04d7086cf7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:42.897726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8999734c-bb9e-4323-a069-fa04d7086cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:42.903149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:42.914170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86438-078d-44ef-ab58-ad5ab27161de admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:42.987663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1057/2094] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:42.991400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:42.991400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:42.991400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:43.024766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:43.025591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:43.026031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:43.026477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] PATH_INFO: `/v3/projects/8b6c8e6074194b42b65a43d6fb10fca5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:43.029403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] MFA Rules not processed for user `baba9ef2b1234fb8a3fed5ae472f8195`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:43.031178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6c8e6074194b42b65a43d6fb10fca5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:43.034357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:43.036760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.122446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6c8e6074194b42b65a43d6fb10fca5', 'request_id': 'req-b5b90e8e-04c8-4a0f-a9b8-93c706287992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.126099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:43.135095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1058/2095] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:43.135949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:43.141130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6c8e6074194b42b65a43d6fb10fca5', 'request_id': 'req-b5b90e8e-04c8-4a0f-a9b8-93c706287992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.146757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.149410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6c8e6074194b42b65a43d6fb10fca5', 'request_id': 'req-b5b90e8e-04c8-4a0f-a9b8-93c706287992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.167571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] The token cache is being invalidate because project 8b6c8e6074194b42b65a43d6fb10fca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:43.168498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.169111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.171147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1038/2096] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:42 2022] DELETE /identity/v3/projects/8b6c8e6074194b42b65a43d6fb10fca5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:43.174322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:43.175199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.221107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a12b1-fcd9-4915-8e43-2954714c7be0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:43.242970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b90e8e-04c8-4a0f-a9b8-93c706287992 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:43.247421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:43.248027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:43.248313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:43.248712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:43.250493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.276576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:43.277268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:43.277586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:43.277876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:43.279896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.290472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:43.326969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:43.329633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:43.342123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1039/2097] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:43.342987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:43.345254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:43.368681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:43.370145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:43.384998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1059/2098] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:43.460509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225f33e-5304-401f-a07e-f0b76e316410 service glance] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:43.467660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:43.468793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:43.469178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:43.469624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] PATH_INFO: `/v3/users/90982b112de04e76830b46f5e5e2479f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:43.471478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90982b112de04e76830b46f5e5e2479f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:43.474841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:43.584400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:43.600571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90982b112de04e76830b46f5e5e2479f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.609975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90982b112de04e76830b46f5e5e2479f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.611081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90982b112de04e76830b46f5e5e2479f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.611949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90982b112de04e76830b46f5e5e2479f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.626648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17f557e8-ea86-40a0-a784-b0c39df7f62a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90982b112de04e76830b46f5e5e2479f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:43.673714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1040/2099] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:43 2022] DELETE /identity/v3/users/90982b112de04e76830b46f5e5e2479f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:44.179678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:44.181343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.201422 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cece0e-ea00-4174-bd3a-63631170e3b0 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:44.393864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:44.394767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.395060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.396131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] PATH_INFO: `/v3/projects/d622b32100ab44de8fad1e0beb754a0e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.399519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d622b32100ab44de8fad1e0beb754a0e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:44.403831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:44.491340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd622b32100ab44de8fad1e0beb754a0e', 'request_id': 'req-b23a4dc4-478e-40c2-aefa-21515e6f6411'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.500245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd622b32100ab44de8fad1e0beb754a0e', 'request_id': 'req-b23a4dc4-478e-40c2-aefa-21515e6f6411'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.510217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd622b32100ab44de8fad1e0beb754a0e', 'request_id': 'req-b23a4dc4-478e-40c2-aefa-21515e6f6411'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.526510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] The token cache is being invalidate because project d622b32100ab44de8fad1e0beb754a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:44.526811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.528775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.535728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1060/2100] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:44 2022] DELETE /identity/v3/projects/d622b32100ab44de8fad1e0beb754a0e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:44.541330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a72d3-a6cc-4f28-a5d3-b43bec9b569f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.541882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a72d3-a6cc-4f28-a5d3-b43bec9b569f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.541882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a72d3-a6cc-4f28-a5d3-b43bec9b569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.576896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:44.584465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b05a72d3-a6cc-4f28-a5d3-b43bec9b569f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:44.585695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b05a72d3-a6cc-4f28-a5d3-b43bec9b569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.586369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.660514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:44.691902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1041/2101] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:44.692131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4dc4-478e-40c2-aefa-21515e6f6411 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:44.693580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.693924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.694324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.695994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:44.696730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.697124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.697677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] PATH_INFO: `/v3/projects/dfcb40889a4b4b888a48b97548fffb09` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.698968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcb40889a4b4b888a48b97548fffb09)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:44.701527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:44.721845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:44.723423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.738864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1042/2102] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:44.743823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:44.757414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77161ab-62c2-4fc7-af60-35e23e550338 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:44.767572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:44.769074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.769873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.771116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.772607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:44.773466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:44.775906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:44.783120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcb40889a4b4b888a48b97548fffb09', 'request_id': 'req-13419fed-5758-4c86-8211-12fb06dab0dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.794565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1043/2103] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:44.795304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcb40889a4b4b888a48b97548fffb09', 'request_id': 'req-13419fed-5758-4c86-8211-12fb06dab0dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.799374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:44.809943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcb40889a4b4b888a48b97548fffb09', 'request_id': 'req-13419fed-5758-4c86-8211-12fb06dab0dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.811870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e245d-1371-4dde-a44e-ad38bfea728b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:44.834760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] The token cache is being invalidate because project dfcb40889a4b4b888a48b97548fffb09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:44.835632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.836350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-13419fed-5758-4c86-8211-12fb06dab0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:44.838123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1061/2104] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:44 2022] DELETE /identity/v3/projects/dfcb40889a4b4b888a48b97548fffb09 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:44.845114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67190908-31c5-4a3e-a170-fd0f0d707410 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.845379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67190908-31c5-4a3e-a170-fd0f0d707410 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.845662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67190908-31c5-4a3e-a170-fd0f0d707410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.873941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-67190908-31c5-4a3e-a170-fd0f0d707410 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:44.876130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67190908-31c5-4a3e-a170-fd0f0d707410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.886471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:44.887700 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.887982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.888258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.888898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:44.892076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:44.929121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1044/2105] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:44.930350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:44.933183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:44.963002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1062/2106] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:44.966390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:44.972118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:44.972386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:44.972818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:44.994795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:44.997211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:45.032666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3a78d-435b-4a97-bd46-d26ebe89833c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.037987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.039302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.039302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.039516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.040853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.042341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.069208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1045/2107] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:45.072986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.075273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1063/2108] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:45.079365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.087147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c9be-ee15-4d6a-88cf-9d837175fa1d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.089790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36434250-9ad0-4c16-b17c-c176645130d6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.092296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.093052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.093387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.093622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.094283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.096136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.097370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.098158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.098607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.098884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.099572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:45.100002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.102120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.104005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:45.115661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1046/2109] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:45.117726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.127137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1064/2110] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:18:45.138659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.144756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932964e6-76e1-4d38-b064-6fe9e484a92e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.150455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.151592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.151742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83178e-55a8-41fc-9a80-f6264066e547 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.152055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.153169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.153169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.155375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.163304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.163947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.164242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.164521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.165175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.167824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.173028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1047/2111] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:45.174040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.188413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872c4c-f9c2-4136-b6d1-4f66f7581d31 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.199047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.200804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.201410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.201868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] PATH_INFO: `/v3/projects/280dfa8a14794440aeac6c75a1c9d471/users/b62f9fe4e66e42e2b3d75bee462ec079/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.226292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1065/2112] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:45 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:45.228138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.231915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=280dfa8a14794440aeac6c75a1c9d471, user_id=b62f9fe4e66e42e2b3d75bee462ec079, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.234780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc432865-2e3f-40e3-907b-f00a76d7abc3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.255517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1048/2113] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:45 2022] PUT /identity/v3/projects/280dfa8a14794440aeac6c75a1c9d471/users/b62f9fe4e66e42e2b3d75bee462ec079/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:45.265796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.266157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.266506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.335418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] MFA Rules not processed for user `b62f9fe4e66e42e2b3d75bee462ec079`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:45.336950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:45.355867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350cd40-3e92-44da-9527-d3a042e5729a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.442484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1049/2114] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:45.457100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.459203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:45.466228 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.466852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.467632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.467927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.468612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.471400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.534633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1066/2115] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:45 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:45.544202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.568409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ae13-7e41-42b8-88bd-75e93b44fafb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.584168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.584802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.585511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.585795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.586651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.588329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.612134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400029c-1e75-433d-a5d7-c451d7277747 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.612681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1067/2116] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:45.614165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.616108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.616737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.617166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.617497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.635300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b8350-f2d2-402c-ab81-e6f39f8bdb3b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.639117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:45.645989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.651612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.652342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.652693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.652801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.653044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.653851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.655702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59871fdf-c805-4989-86cc-41a05d508c2e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.687847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1050/2117] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:45.695163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.701282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1068/2118] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:45.717867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d1aa-44ac-4490-a3f8-333b376de4d2 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:45.725087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:45.726309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.726760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.727259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] PATH_INFO: `/v3/projects/43a1fa996402448091ef05ed5b8467af/users/af14713dfbb348fa8824570570225de4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.748406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a1fa996402448091ef05ed5b8467af, user_id=af14713dfbb348fa8824570570225de4, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:45.753469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:45.780129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1051/2119] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:45 2022] PUT /identity/v3/projects/43a1fa996402448091ef05ed5b8467af/users/af14713dfbb348fa8824570570225de4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:45.785602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:45.785942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:45.786680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:45.810302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:45.814374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:45.920013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:45.951772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1069/2120] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:45.962820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:45.965494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:46.049593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094aff8-53f0-40ca-9414-7b4bf334653d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:46.053530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:46.054384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:46.055089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:46.056530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:46.074291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:46.078967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:46.093524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1052/2121] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:46.213813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:46.215014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:46.234225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd468b03-0ac8-4b4e-b21f-3654b76b2e2e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:46.238871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:46.239900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:46.240225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:46.240439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] PATH_INFO: `/v3/users/ca6d51cbceff489bae4222ff8888ac16` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:46.241865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6d51cbceff489bae4222ff8888ac16)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:46.244487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:46.305012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6d51cbceff489bae4222ff8888ac16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.312875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6d51cbceff489bae4222ff8888ac16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.313561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6d51cbceff489bae4222ff8888ac16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.321834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6d51cbceff489bae4222ff8888ac16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.336462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a1d96350-a530-4a1d-977f-c4419d309709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6d51cbceff489bae4222ff8888ac16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.343210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1070/2122] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:46 2022] DELETE /identity/v3/users/ca6d51cbceff489bae4222ff8888ac16 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:46.664108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:46.676615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0b5b2-76b4-4e8d-96dd-6771ccc145a7 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:46.774033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:46.774944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:46.775512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:46.776017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] PATH_INFO: `/v3/projects/6c148946e0fe4eadacb3cfcade050e5f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:46.777282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c148946e0fe4eadacb3cfcade050e5f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:46.779458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:46.834156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c148946e0fe4eadacb3cfcade050e5f', 'request_id': 'req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.840814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c148946e0fe4eadacb3cfcade050e5f', 'request_id': 'req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.846673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c148946e0fe4eadacb3cfcade050e5f', 'request_id': 'req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.864497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] The token cache is being invalidate because project 6c148946e0fe4eadacb3cfcade050e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:46.864797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.865550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb8a0a0-d1a4-40ef-bde8-2a502c185103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:46.867475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1053/2123] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:46 2022] DELETE /identity/v3/projects/6c148946e0fe4eadacb3cfcade050e5f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:46.872782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:46.873189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:46.873559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:46.903113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:46.904493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:47.023789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1071/2124] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:47.031171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.031484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.031757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.049776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:47.078037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:47.079548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:47.095746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1054/2125] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:47.096761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.110637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56414784-3361-40e9-aa30-9bc0a29432cf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.116550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.116968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.117997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.117997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.118783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:47.119084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.121742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.149585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1072/2126] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:18:47.153630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.166431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025579fc-7e47-43fb-8051-85345a094fe8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.175329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.176563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.177000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.177442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.178446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.181164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.221904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1055/2127] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:47.226183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.237412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f901c1-8df9-4d48-8b69-08c4f403328c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.295460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:47.323419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.324130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.324447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.324717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.325373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.327983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.359835 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1073/2128] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/users => generated 303 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:47.364086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.379638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bb9f1-97b1-4160-8e24-548731ed5f9f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.385666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.386903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.386903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.387124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.388221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.389966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.401587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1056/2129] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:47.410465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.421796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b263-6157-4529-980a-5948ad69b7f8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.426593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.427587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.427587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.427767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.428412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.430050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf608a-4f6d-4ff4-a551-09a98d62f1d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.447099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1074/2130] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:47.448104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.462435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbb5ed-d775-486c-9abc-7b2aef4889d6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.470247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.471026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.471386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.471659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] PATH_INFO: `/v3/projects/531cf110a3f144cc98a89c482ebadd46/users/aba0eecd97414d50a354c7975da18171/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.511946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] RBAC: Authorizing `identity:create_grant(project_id=531cf110a3f144cc98a89c482ebadd46, user_id=aba0eecd97414d50a354c7975da18171, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.514810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.533668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1057/2131] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:47 2022] PUT /identity/v3/projects/531cf110a3f144cc98a89c482ebadd46/users/aba0eecd97414d50a354c7975da18171/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:47.549897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.550185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.550462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.583900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] MFA Rules not processed for user `aba0eecd97414d50a354c7975da18171`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:47.585298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:47.691406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1075/2132] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:47.696095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.780528 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc7d7-340c-48fc-9c4d-edae583d051d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.787104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.787783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.788036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.788379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.789462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.792243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.853162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1058/2133] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/projects => generated 403 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:47.858832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:47.875544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288c834-1213-46c2-93de-3b5bcf66d2d0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:47.951522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:47.952204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:47.952489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:47.952789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:47.953499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:47.955813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:47.983113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1076/2134] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:47 2022] POST /identity/v3/users => generated 308 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:47.988535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.004898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea998f1-3a48-444a-8a69-1d6e9927a45d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.013388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.014060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.014248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.014527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.015904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.018048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:48.036485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1059/2135] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:48.039983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.055460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfe593-f322-4078-bce4-c4a45b68967f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.071464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.073169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.073610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.074116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] PATH_INFO: `/v3/projects/d11d78b83a384fcbb143fdf41656e1b9/users/75d933e0ca9746f7ba9c653f14a5b0bd/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.095495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11d78b83a384fcbb143fdf41656e1b9, user_id=75d933e0ca9746f7ba9c653f14a5b0bd, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.098281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:48.122487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1077/2136] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:48 2022] PUT /identity/v3/projects/d11d78b83a384fcbb143fdf41656e1b9/users/75d933e0ca9746f7ba9c653f14a5b0bd/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:48.131905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.146693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f2f5-51e9-4f2b-87eb-d98df0a7675f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.225004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.225631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.225917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.227807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.228500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.230527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64916-652d-4692-9033-3dd8f83c577d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:48.244298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1060/2137] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:48.248987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.266247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f977eb-9f57-44b7-9ad2-5bd064415614 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.273873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.274709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.274999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.276030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] PATH_INFO: `/v3/projects/d11d78b83a384fcbb143fdf41656e1b9/users/75d933e0ca9746f7ba9c653f14a5b0bd/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.278620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11d78b83a384fcbb143fdf41656e1b9, user_id=75d933e0ca9746f7ba9c653f14a5b0bd, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.282597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536a54d-cb6f-4107-8187-118e04fd8520 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:48.299932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1078/2138] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:48 2022] PUT /identity/v3/projects/d11d78b83a384fcbb143fdf41656e1b9/users/75d933e0ca9746f7ba9c653f14a5b0bd/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:48.313290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.313641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.313977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.317315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.318044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.318650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.342817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:48.344413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:48.352313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] MFA Rules not processed for user `b62f9fe4e66e42e2b3d75bee462ec079`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:48.354409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:48.354659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] MFA Rules not processed for user `75d933e0ca9746f7ba9c653f14a5b0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:48.357610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:48.437863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1079/2139] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:48.478424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1061/2140] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:48.485380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.486941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:48.570493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490149ec-0a01-4df0-bdfc-c158ee8d5096 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.575031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.575866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.576548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.576926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.597711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.599381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46b90-3d16-40f8-8039-5b7d648c4afc service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:48.611532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1080/2141] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:48 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:18:48.794267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:48.906499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18691d39-43f0-462a-a83c-e0fcceb2b162 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:48.920751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:48.922268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:48.922663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:48.923678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] PATH_INFO: `/v3/users/aba0eecd97414d50a354c7975da18171` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:48.926240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba0eecd97414d50a354c7975da18171)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:48.928265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:49.039456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba0eecd97414d50a354c7975da18171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:49.054232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba0eecd97414d50a354c7975da18171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:49.055506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba0eecd97414d50a354c7975da18171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:49.059673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba0eecd97414d50a354c7975da18171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:49.080612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba0eecd97414d50a354c7975da18171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:49.140478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1062/2142] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:48 2022] DELETE /identity/v3/users/aba0eecd97414d50a354c7975da18171 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:49.145502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:49.145833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:49.146113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:49.220072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:49.221617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:49.315595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1081/2143] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:49.328317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:49.329927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:49.376326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:49.486176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f87376-805a-484f-a920-710c35f37350 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:49.494100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:49.494577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:49.494864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:49.495119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:49.514484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:49.516052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:49.540102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1063/2144] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:50.818433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:50.821334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:50.850180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bf03f-2a02-41a8-938d-d00e266661a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:50.862346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:50.863298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:50.863791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:50.864267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] PATH_INFO: `/v3/projects/531cf110a3f144cc98a89c482ebadd46` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:50.865698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=531cf110a3f144cc98a89c482ebadd46)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:50.867877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:50.946292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531cf110a3f144cc98a89c482ebadd46', 'request_id': 'req-aa307ea9-262e-42b3-9769-82e82d45e470'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:50.949668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:50.965803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531cf110a3f144cc98a89c482ebadd46', 'request_id': 'req-aa307ea9-262e-42b3-9769-82e82d45e470'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:50.989541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531cf110a3f144cc98a89c482ebadd46', 'request_id': 'req-aa307ea9-262e-42b3-9769-82e82d45e470'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.010889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] The token cache is being invalidate because project 531cf110a3f144cc98a89c482ebadd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:51.012167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.013031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.018084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1082/2145] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:50 2022] DELETE /identity/v3/projects/531cf110a3f144cc98a89c482ebadd46 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:51.021391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:51.045108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e77b1-58c9-4146-bc93-2aa7cd007aab service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:51.170228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:51.171196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:51.171609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:51.172089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] PATH_INFO: `/v3/users/75d933e0ca9746f7ba9c653f14a5b0bd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:51.173425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d933e0ca9746f7ba9c653f14a5b0bd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:51.175813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:51.258169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d933e0ca9746f7ba9c653f14a5b0bd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.271814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d933e0ca9746f7ba9c653f14a5b0bd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.272934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d933e0ca9746f7ba9c653f14a5b0bd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.273593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d933e0ca9746f7ba9c653f14a5b0bd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.284493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4a9653fb-fae0-43b5-905b-646e04b9aa19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d933e0ca9746f7ba9c653f14a5b0bd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:51.336061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1064/2146] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:51 2022] DELETE /identity/v3/users/75d933e0ca9746f7ba9c653f14a5b0bd => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:18:51.899696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:51.903597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:51.950108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307ea9-262e-42b3-9769-82e82d45e470 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:52.087834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:52.088486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.088949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.089252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] PATH_INFO: `/v3/projects/d11d78b83a384fcbb143fdf41656e1b9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.092009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11d78b83a384fcbb143fdf41656e1b9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:52.094710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:52.166862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11d78b83a384fcbb143fdf41656e1b9', 'request_id': 'req-a5af3a78-80fe-4723-9adb-3f6727a77077'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:52.176528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11d78b83a384fcbb143fdf41656e1b9', 'request_id': 'req-a5af3a78-80fe-4723-9adb-3f6727a77077'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:52.185983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11d78b83a384fcbb143fdf41656e1b9', 'request_id': 'req-a5af3a78-80fe-4723-9adb-3f6727a77077'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:52.202280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] The token cache is being invalidate because project d11d78b83a384fcbb143fdf41656e1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:52.202860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:52.203614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5af3a78-80fe-4723-9adb-3f6727a77077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:52.205160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1083/2147] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:51 2022] DELETE /identity/v3/projects/d11d78b83a384fcbb143fdf41656e1b9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:52.210690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.211175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.211303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.236912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:52.238224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:52.293965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:52.360364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1065/2148] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:52.363484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.366705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.366705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.395661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:52.398293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:52.416217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1084/2149] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:52.420939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:52.437337 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aad06-455c-4ba8-8888-4d2f2012da5d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:52.456521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:52.457281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.457702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.458162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.459140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:52.459670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:52.461366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:52.492865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1066/2150] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:18:52.498790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:52.518186 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493601-0922-41c5-bf42-20cfddbea310 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:52.540336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:52.541135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.549195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.549878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.551276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:52.567391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352243d-d17b-4ba8-957c-5ae35ddbdfa2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:52.624345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1085/2151] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:52 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:52.625401 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:52.643786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9668050-025c-4754-819b-891a5e43dc01 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:52.734838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.735600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.736081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.780336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] MFA Rules not processed for user `af14713dfbb348fa8824570570225de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:52.783486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:52.820387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:52.821062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.821339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.821607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.822254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:52.824827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:52.897974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1067/2152] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:52 2022] POST /identity/v3/users => generated 286 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:52.907741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:52.926851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:52.956966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8b1b-08e6-4e29-b019-350974ae80b8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:52.986579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:52.987119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:52.987588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:52.991040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:52.992292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.001785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.020602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1068/2153] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:53.021673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.024993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1086/2154] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:53.030677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.034862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:53.039142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fb3a-c75c-4352-999e-7193584677bc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.046187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.047215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.047914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.048606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.055253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.059050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.060092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca576f-cb47-4ba4-a0b2-90bb100cfe25 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.075342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.078877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.078877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.079722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.083261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1069/2155] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:53.083712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.085591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.088567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.099062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263887e-f5f4-4cd2-9a53-be41e9d65888 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.104776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.105720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.106200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.106328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] PATH_INFO: `/v3/projects/c9474141577e47c1b04cf8ba37dd9241/users/2d2abd1ad291487383f88b7d028136f0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.139670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9474141577e47c1b04cf8ba37dd9241, user_id=2d2abd1ad291487383f88b7d028136f0, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.147570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1087/2156] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:53 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:53.149131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.150091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c255c6-98d9-4320-8235-5b2ee2799112 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.178531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1070/2157] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:53 2022] PUT /identity/v3/projects/c9474141577e47c1b04cf8ba37dd9241/users/2d2abd1ad291487383f88b7d028136f0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:53.189884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.190355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.191291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.240415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] MFA Rules not processed for user `2d2abd1ad291487383f88b7d028136f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:53.243293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:53.284827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c0d5-206f-4738-b309-6d834c8dc699 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.380017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:53.391297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1071/2158] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:18:53.392610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.408567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.409213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.409543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.409871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.410582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.413330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.452049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1088/2159] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:53 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:53.456601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.489433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dfe04-01c8-4c13-82d8-50c19b65eabb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.506498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.507072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.507543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.507823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.508491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.510117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.548426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1089/2160] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:53.554814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.569339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d999fae-624b-425a-bf24-43f8ddd348b8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.574137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26ad5-3b16-4236-818f-6d00479f5c36 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.575892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.576790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.577318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.577505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.578120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.583644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.585736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.586255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.586621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] PATH_INFO: `/v3/projects/23ca2d1ee84e4a959766d2954348a778/users/ef855b112268459e899e30da4f5d2cf7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.590573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.634614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ca2d1ee84e4a959766d2954348a778, user_id=ef855b112268459e899e30da4f5d2cf7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.653201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.684872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1090/2161] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:53 2022] PUT /identity/v3/projects/23ca2d1ee84e4a959766d2954348a778/users/ef855b112268459e899e30da4f5d2cf7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:53.685724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1072/2162] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:53 2022] POST /identity/v3/projects => generated 373 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:53.686493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.690270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.716233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:53.851345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870dcc-a561-4084-9fa2-32847ce5a1d5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.866302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.867323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.867732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.868143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.869606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.875261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.890299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff94cd8-4cfe-483d-9e14-d581d5e4d920 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.907061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.910213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.910523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.911982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.913243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.919144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:53.923448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1073/2163] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:53 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:53.930296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.959968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1091/2164] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:53.961621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:53.971706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f7a8-929f-42a1-9ce6-2a838edaea6f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.974410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463b016-42b3-4b9e-87bb-ab80821dd19c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:53.980464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.981524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.981998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.985754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] PATH_INFO: `/v3/projects/23ca2d1ee84e4a959766d2954348a778/users/ef855b112268459e899e30da4f5d2cf7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.986631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:53.987333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:53.987620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:53.987896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:53.988978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.992068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ca2d1ee84e4a959766d2954348a778, user_id=ef855b112268459e899e30da4f5d2cf7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:53.993304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.001676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.017739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1074/2165] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:54.022254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:54.031326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1092/2166] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:53 2022] PUT /identity/v3/projects/23ca2d1ee84e4a959766d2954348a778/users/ef855b112268459e899e30da4f5d2cf7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:54.038262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b975e00-e63a-4a3c-870c-3ccbdc1ee8fb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:54.209432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:54.211001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:54.211466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:54.211908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] PATH_INFO: `/v3/projects/71f116d6b0c6400da233fa0010931807/users/e7f20cd4edcf4a2498400ef13a3ac6a8/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:54.217726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:54.242116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f116d6b0c6400da233fa0010931807, user_id=e7f20cd4edcf4a2498400ef13a3ac6a8, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:54.249796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.272694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1075/2167] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:54 2022] PUT /identity/v3/projects/71f116d6b0c6400da233fa0010931807/users/e7f20cd4edcf4a2498400ef13a3ac6a8/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:54.273524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:54.293822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88cf5e-6ef5-4042-b542-d440e2f7a3e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:54.479349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:54.481053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:54.484446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:54.484913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:54.486033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:54.487878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75408166-6f7d-4e4c-a79b-0a15dd2da1b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.503100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1093/2168] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:18:54.506824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:54.522647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11519ab6-fece-4db6-88c3-b9f7a0c9ad4e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:54.537531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:54.538225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:54.538613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:54.538775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] PATH_INFO: `/v3/projects/71f116d6b0c6400da233fa0010931807/users/e7f20cd4edcf4a2498400ef13a3ac6a8/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:54.543655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f116d6b0c6400da233fa0010931807, user_id=e7f20cd4edcf4a2498400ef13a3ac6a8, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:54.546597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.581922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1076/2169] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:54 2022] PUT /identity/v3/projects/71f116d6b0c6400da233fa0010931807/users/e7f20cd4edcf4a2498400ef13a3ac6a8/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:54.586196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:54.586748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:54.587656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:54.631039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] MFA Rules not processed for user `e7f20cd4edcf4a2498400ef13a3ac6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:54.636174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:54.759599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1094/2170] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:54.772334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:54.775016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:54.907208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303148-5698-4528-ad90-7774713baaa4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:54.913124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:54.914221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:54.914509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:54.915312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:54.928573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:54.934967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166e354-8855-4148-b241-d905a3e08ddc service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:54.952004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1077/2171] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:55.790209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:55.830614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:55.954801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d281e-eb90-4ba5-8031-e851c2600f23 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:55.962724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:55.964051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:55.964398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:55.965601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] PATH_INFO: `/v3/users/2d2abd1ad291487383f88b7d028136f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:55.966884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2abd1ad291487383f88b7d028136f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:55.972030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:56.037378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2abd1ad291487383f88b7d028136f0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:56.047946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2abd1ad291487383f88b7d028136f0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:56.048790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2abd1ad291487383f88b7d028136f0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:56.061681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2abd1ad291487383f88b7d028136f0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:56.120002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2abd1ad291487383f88b7d028136f0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:56.136998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1095/2172] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:55 2022] DELETE /identity/v3/users/2d2abd1ad291487383f88b7d028136f0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:56.143637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:56.144722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:56.145153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:56.167245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:56.169545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:56.306339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1078/2173] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:56.335960 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:56.340453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:56.499932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fc4da-51dd-4173-97de-9f70d0283807 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:56.514930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:56.515608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:56.515897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:56.516172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:56.539987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:56.541483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:56.563786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1096/2174] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:18:57.460485 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:57.462403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:57.484314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3abef-ad91-41e3-b362-46e7d9ca35da None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:57.504941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:57.505603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:57.510696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:57.521026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] PATH_INFO: `/v3/projects/c9474141577e47c1b04cf8ba37dd9241` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:57.528631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9474141577e47c1b04cf8ba37dd9241)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:57.537430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:57.646205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9474141577e47c1b04cf8ba37dd9241', 'request_id': 'req-15036956-7a4f-467a-8572-51670e8dee51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:57.661221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9474141577e47c1b04cf8ba37dd9241', 'request_id': 'req-15036956-7a4f-467a-8572-51670e8dee51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:57.668875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9474141577e47c1b04cf8ba37dd9241', 'request_id': 'req-15036956-7a4f-467a-8572-51670e8dee51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:57.681280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:57.717178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] The token cache is being invalidate because project c9474141577e47c1b04cf8ba37dd9241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:57.721809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:57.723073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:57.726104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1079/2175] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:57 2022] DELETE /identity/v3/projects/c9474141577e47c1b04cf8ba37dd9241 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:57.729417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:57.748027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd24c48-92c2-42cc-8aca-aeaf08a2c3ef service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:57.815121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:57.872165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:57.872946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:57.873250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:57.873557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] PATH_INFO: `/v3/users/e7f20cd4edcf4a2498400ef13a3ac6a8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:57.875312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f20cd4edcf4a2498400ef13a3ac6a8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:57.880334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:57.994040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f20cd4edcf4a2498400ef13a3ac6a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.008066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f20cd4edcf4a2498400ef13a3ac6a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.021774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f20cd4edcf4a2498400ef13a3ac6a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.046892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f20cd4edcf4a2498400ef13a3ac6a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.094166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1cad849-0e59-4288-8979-75b913a63c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f20cd4edcf4a2498400ef13a3ac6a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.112463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1097/2176] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:18:57 2022] DELETE /identity/v3/users/e7f20cd4edcf4a2498400ef13a3ac6a8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:58.558193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:58.559745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:58.591281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15036956-7a4f-467a-8572-51670e8dee51 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:58.696062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:58.697021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:58.697457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:58.699583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] PATH_INFO: `/v3/projects/71f116d6b0c6400da233fa0010931807` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:58.704483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f116d6b0c6400da233fa0010931807)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:58.706885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:58.853896 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f116d6b0c6400da233fa0010931807', 'request_id': 'req-11e58d56-7c02-4a47-8a6b-12f70683b84e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.865416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f116d6b0c6400da233fa0010931807', 'request_id': 'req-11e58d56-7c02-4a47-8a6b-12f70683b84e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.884425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f116d6b0c6400da233fa0010931807', 'request_id': 'req-11e58d56-7c02-4a47-8a6b-12f70683b84e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.906030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] The token cache is being invalidate because project 71f116d6b0c6400da233fa0010931807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:18:58.906937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.914587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11e58d56-7c02-4a47-8a6b-12f70683b84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:18:58.920163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1080/2177] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:18:58 2022] DELETE /identity/v3/projects/71f116d6b0c6400da233fa0010931807 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:58.921840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:58.922297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:58.922390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:58.965825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:58.967076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:59.089157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1098/2178] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:59.102681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.102975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.103273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.141515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:59.143256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:59.159717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1081/2179] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:59.164900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.176724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438c907-7f2e-4eaf-bb4f-0dfc8d57a9b4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.185059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.186605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.187139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.187758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.189240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:18:59.198055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.200696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.226392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1099/2180] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:18:59.228017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.238202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e4b5-ad13-480b-851b-4a1352ce2135 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.243437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.244060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.244341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.244623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.245286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.247826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.287016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1082/2181] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:59.291389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.301668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec696b-5618-46c4-9b1e-e5dcf44c21ee admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.354535 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:59.379701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.380760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.381256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.381720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.382798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.385386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.417136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1100/2182] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:18:59.421137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.432959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a01-90a9-4656-bd30-b65131a5711b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.441596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.442434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.442789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.443135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.444789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.447510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.471034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1083/2183] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:59.474873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.528253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f38a9-02db-45b9-a352-ece3cb7aa5c5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.536338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.538111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.538718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.539276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.554996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.566127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11143282-5db1-4c4b-9c3b-de06de3d507a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.604417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1101/2184] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:18:59.606935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.620695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdba89c-e0a7-4157-b2bf-82914ff81094 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.632186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.632897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.633179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.633462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] PATH_INFO: `/v3/projects/f0dcc7e3d1e441eba08ffbb715476171/users/91dea83320b147fa91158a59e53d70c9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.650802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0dcc7e3d1e441eba08ffbb715476171, user_id=91dea83320b147fa91158a59e53d70c9, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.653573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.673296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1084/2185] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:18:59 2022] PUT /identity/v3/projects/f0dcc7e3d1e441eba08ffbb715476171/users/91dea83320b147fa91158a59e53d70c9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:18:59.685051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.685395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.686042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.717203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] MFA Rules not processed for user `91dea83320b147fa91158a59e53d70c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:18:59.718457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:18:59.811394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1102/2186] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:18:59.813750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.849258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:18:59.900387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9535a04-6c16-4315-aac1-5389a96556ab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:18:59.905980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:18:59.906595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:18:59.906877 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:18:59.907379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:18:59.908319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:18:59.910268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:18:59.944222 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1085/2187] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/projects => generated 391 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:18:59.947313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:18:59.959123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609015-a9eb-47ab-86ea-c5aed442494b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.045104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.045886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.046177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.046464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.047112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.049113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.085470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1103/2188] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:18:59 2022] POST /identity/v3/users => generated 302 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:19:00.086182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.101415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27986d3-d076-4387-bab6-f9b997a4fd69 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.115347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.116145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.116437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.116724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.117382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.119205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.138901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1086/2189] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:00.150599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.179949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f2a3a-abb6-45ec-91b7-4cb4ec66d01e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.186553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.187477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.187768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.188536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] PATH_INFO: `/v3/projects/7e10d95e495345ed950658966d5bae7c/users/fd05a37d86ce44209028b6d586e60e78/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.209152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e10d95e495345ed950658966d5bae7c, user_id=fd05a37d86ce44209028b6d586e60e78, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.212783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.240218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1104/2190] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:00 2022] PUT /identity/v3/projects/7e10d95e495345ed950658966d5bae7c/users/fd05a37d86ce44209028b6d586e60e78/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:00.241920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.260040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041c03-6622-42a1-82ce-3f893f1486c0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.450439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.451862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.452143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.452466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.453187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.455924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced2300-3ca6-48e2-af54-60c0b2f8e547 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.504498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1087/2191] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:00.508450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.521464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac4646-ba62-461c-a9e1-6fe8abbe1486 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.529562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.530378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.530699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.531050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] PATH_INFO: `/v3/projects/7e10d95e495345ed950658966d5bae7c/users/fd05a37d86ce44209028b6d586e60e78/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.540884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e10d95e495345ed950658966d5bae7c, user_id=fd05a37d86ce44209028b6d586e60e78, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.545102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.561797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1105/2192] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:00 2022] PUT /identity/v3/projects/7e10d95e495345ed950658966d5bae7c/users/fd05a37d86ce44209028b6d586e60e78/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:00.572122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.572440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.572909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.625662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] MFA Rules not processed for user `fd05a37d86ce44209028b6d586e60e78`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:00.628161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:00.752289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1088/2193] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:00.762852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.765356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:00.772072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:00.856954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d754a8-f6b8-481f-a723-ba04778100b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.864520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.865055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.865334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.866124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:00.895496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:00.897008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a281c-21b0-4958-a9d0-df55ebcba2ec service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:00.912160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1106/2194] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:00 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:00.965949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:00.968783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:00.982205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498c9c2-1d1c-4c3f-949c-08eda51dcbcd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:00.986433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:00.987125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:00.987521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:00.987872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:01.019132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:01.073260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:01.075479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:01.086646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1089/2195] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:02.366966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc5f4e-15fc-452d-b47b-c81674d49735 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:02.367502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc5f4e-15fc-452d-b47b-c81674d49735 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:02.367709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc5f4e-15fc-452d-b47b-c81674d49735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:02.404833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a8fc5f4e-15fc-452d-b47b-c81674d49735 None None] MFA Rules not processed for user `ef855b112268459e899e30da4f5d2cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:02.406260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a8fc5f4e-15fc-452d-b47b-c81674d49735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:02.481166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1107/2196] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:02.495073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:02.498221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:02.578246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98cf7-171d-47eb-bff6-5278836195fc service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:02.583780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:02.584383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:02.584665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:02.584941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:02.586633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:02.649059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:02.650681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:02.660423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1090/2197] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:04.036329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e38ab3-00f6-4885-8cda-51c15d9d9e90 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:04.036898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e38ab3-00f6-4885-8cda-51c15d9d9e90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:04.037086 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e38ab3-00f6-4885-8cda-51c15d9d9e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:04.068717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f6e38ab3-00f6-4885-8cda-51c15d9d9e90 None None] MFA Rules not processed for user `af14713dfbb348fa8824570570225de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:04.069923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f6e38ab3-00f6-4885-8cda-51c15d9d9e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:04.078133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1108/2198] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:04.904952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:04.985957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:05.001379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70328989-9758-4672-b1e0-221c8eb29e66 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:05.007094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:05.008340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:05.008659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:05.008951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] PATH_INFO: `/v3/users/91dea83320b147fa91158a59e53d70c9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:05.010185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dea83320b147fa91158a59e53d70c9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:05.013484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:05.071776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dea83320b147fa91158a59e53d70c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.080396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dea83320b147fa91158a59e53d70c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.081134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dea83320b147fa91158a59e53d70c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.081754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dea83320b147fa91158a59e53d70c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.091681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dea83320b147fa91158a59e53d70c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.112285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1091/2199] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:04 2022] DELETE /identity/v3/users/91dea83320b147fa91158a59e53d70c9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:05.117337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:05.117897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:05.117897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:05.138981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:05.140222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:05.215908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1109/2200] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:05.226848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:05.228318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:05.291571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34473-b8dc-4b87-a317-ae913b8480e6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:05.295727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:05.296434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:05.296804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:05.297086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:05.310231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:05.311449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:05.324900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1092/2201] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:05.718523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:05.720011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:05.743381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3227d5-663f-48ed-9747-3542dbf79dd6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:05.749288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:05.750305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:05.750571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:05.750852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] PATH_INFO: `/v3/projects/f0dcc7e3d1e441eba08ffbb715476171` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:05.752446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0dcc7e3d1e441eba08ffbb715476171)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:05.754447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:05.778530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:05.811457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0dcc7e3d1e441eba08ffbb715476171', 'request_id': 'req-21ac0864-2bc5-4c05-9814-99cbf037d472'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.823034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0dcc7e3d1e441eba08ffbb715476171', 'request_id': 'req-21ac0864-2bc5-4c05-9814-99cbf037d472'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.830569 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0dcc7e3d1e441eba08ffbb715476171', 'request_id': 'req-21ac0864-2bc5-4c05-9814-99cbf037d472'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.843511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] The token cache is being invalidate because project f0dcc7e3d1e441eba08ffbb715476171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:05.843748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.844403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:05.846864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1110/2202] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:05 2022] DELETE /identity/v3/projects/f0dcc7e3d1e441eba08ffbb715476171 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:05.852678 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:05.863478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdfc3-8155-42d9-b665-3679747fdf0e service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:05.928604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:05.929235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:05.929445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:05.929749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] PATH_INFO: `/v3/users/fd05a37d86ce44209028b6d586e60e78` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:05.930955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd05a37d86ce44209028b6d586e60e78)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:05.933525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:05.957571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:05.989904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd05a37d86ce44209028b6d586e60e78'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.002464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd05a37d86ce44209028b6d586e60e78'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.004166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd05a37d86ce44209028b6d586e60e78'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.004984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd05a37d86ce44209028b6d586e60e78'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.014342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87129b6d-981a-4dcc-bce3-ee150941917c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd05a37d86ce44209028b6d586e60e78'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.035655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1093/2203] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:05 2022] DELETE /identity/v3/users/fd05a37d86ce44209028b6d586e60e78 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:06.349014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:06.351398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:06.365971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0864-2bc5-4c05-9814-99cbf037d472 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:06.443970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:06.445285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.445719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.446154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] PATH_INFO: `/v3/projects/7e10d95e495345ed950658966d5bae7c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.447626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e10d95e495345ed950658966d5bae7c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:06.452403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:06.512124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e10d95e495345ed950658966d5bae7c', 'request_id': 'req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.521506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e10d95e495345ed950658966d5bae7c', 'request_id': 'req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.528640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e10d95e495345ed950658966d5bae7c', 'request_id': 'req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.557699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] The token cache is being invalidate because project 7e10d95e495345ed950658966d5bae7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:06.558672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.559452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-eb3a8ab8-79f2-4dcf-b646-5241b22fae47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:06.560998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1111/2204] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:06 2022] DELETE /identity/v3/projects/7e10d95e495345ed950658966d5bae7c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:06.566256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.566666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.567122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.598157 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:06.599637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:06.691441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1094/2205] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:06.704331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.704851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.705318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.753656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:06.754970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:06.764844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1112/2206] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:06.769142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:06.779829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc91258-f62a-40bc-95dd-bc7218465034 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:06.784534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:06.785138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.785423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.785702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.786356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:19:06.786659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:06.788287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:06.801115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1095/2207] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:19:06.805559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:06.815125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d272a-6c6d-4de6-a16d-fa857436ddfc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:06.822107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:06.822795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.823183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.823532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.824387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:06.826385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:06.859748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1113/2208] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:06 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:06.860919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:06.871489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffde6e-9491-469c-a6c6-3f32b564d3a6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:06.932716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:06.933463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.933605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.933883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.934537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:06.936498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:06.958878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1096/2209] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:06 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:19:06.962662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:06.976368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba459e-6c7b-4882-943b-d3be9817339a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:06.981826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:06.982431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:06.982716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:06.983009 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:06.983820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:06.985460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:06.987577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:07.002450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1114/2210] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:07.006917 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.017385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46bfd5-42c2-4dac-94ae-249f36504928 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.022029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.022619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.022902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.023178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.023880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.025741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d1e47-46ca-454e-b600-6229b7c08a55 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.038977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1097/2211] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:07.044073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.054248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab4b-205b-4221-9a36-3b67dbc9b920 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.058984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.059667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.060189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.060481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] PATH_INFO: `/v3/projects/e86c5a35cd7246c383df3dc2ede4c8ee/users/74620f7b96004504a40683303175e6b0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.078785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86c5a35cd7246c383df3dc2ede4c8ee, user_id=74620f7b96004504a40683303175e6b0, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.081563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.096082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1115/2212] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:07 2022] PUT /identity/v3/projects/e86c5a35cd7246c383df3dc2ede4c8ee/users/74620f7b96004504a40683303175e6b0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:07.109903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479df3e-b214-422e-b789-69634e43448c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.110230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479df3e-b214-422e-b789-69634e43448c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.110577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479df3e-b214-422e-b789-69634e43448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.117786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:07.141196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5479df3e-b214-422e-b789-69634e43448c None None] MFA Rules not processed for user `74620f7b96004504a40683303175e6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:07.142467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5479df3e-b214-422e-b789-69634e43448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:07.217165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1098/2213] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:07.221771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.284051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4edba-a8d7-4fa4-a64e-c06be5719b68 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.288659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.289251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.289530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.289798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.290418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.292417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.324063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1116/2214] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:19:07.325276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479df3e-b214-422e-b789-69634e43448c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.336262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479df3e-b214-422e-b789-69634e43448c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.395744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.396351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.396635 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.396910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.397540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.399469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.421198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1099/2215] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:07.425270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.435551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa950a2-a900-491e-8bda-e5653685efb4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.440262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.440874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.441157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.441436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.442129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.443729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.455555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1117/2216] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:07.461209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.473474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6dc6-57c0-422c-b327-6ed373c15a8e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.478600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.479199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.479511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.479810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.480453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.482056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb852c-8437-4c14-b68d-0667e4373295 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.497202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1100/2217] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:07.498501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.509646 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf903d-c2b5-458b-b3a2-91452e544adb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.523317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.524252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.524860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.525138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] PATH_INFO: `/v3/projects/13564aa78a054e658cafe535ef3ff698/users/235694e694374d1d919d52369c04ac8e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.549505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13564aa78a054e658cafe535ef3ff698, user_id=235694e694374d1d919d52369c04ac8e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.552288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.566126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1118/2218] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:07 2022] PUT /identity/v3/projects/13564aa78a054e658cafe535ef3ff698/users/235694e694374d1d919d52369c04ac8e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:07.580257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.580632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.581291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.614683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] MFA Rules not processed for user `235694e694374d1d919d52369c04ac8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:07.616081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:07.688727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:07.696655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1101/2219] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:07.697892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.804308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4acbe2-9a42-4451-846d-39b5f7941448 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.810210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.810894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.811310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.811663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.812370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.814378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.845432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1119/2220] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:07.847641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:07.871206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c01f-4840-47f7-b596-1646bf044a68 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:07.953089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:07.953722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:07.954120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:07.954407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:07.955127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:07.957163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:07.984931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1102/2221] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:07 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:07.986558 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.002276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf962df-1bb3-4986-a1e6-3943577e6750 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.008047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.008685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.008972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.009248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.010136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.011751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.016367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:08.035411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1120/2222] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:08.037718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.049964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4ee34-20b3-4041-a73a-6076bf8ddc1f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.055474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.056140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.056425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.056738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] PATH_INFO: `/v3/projects/234a9bec959e4e1681b28596dbe44adb/users/df7633b35b5e42dc9795c28765d17026/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.074671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=234a9bec959e4e1681b28596dbe44adb, user_id=df7633b35b5e42dc9795c28765d17026, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.077844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.094120 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1103/2223] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:08 2022] PUT /identity/v3/projects/234a9bec959e4e1681b28596dbe44adb/users/df7633b35b5e42dc9795c28765d17026/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:08.098413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.110653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77d87-6d7e-402e-a55f-74dcba4047fb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.233280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.233792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.234078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.234355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.235022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.236688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e86abb-cbcf-4094-a723-c1a83d9b6575 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.251315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1121/2224] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:08.252834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.263533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d04a-14ba-4ba7-8f80-dab093dc12cd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.268442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.269134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.269419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.269694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] PATH_INFO: `/v3/projects/234a9bec959e4e1681b28596dbe44adb/users/df7633b35b5e42dc9795c28765d17026/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.271846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234a9bec959e4e1681b28596dbe44adb, user_id=df7633b35b5e42dc9795c28765d17026, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.274491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.285773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1104/2225] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:08 2022] PUT /identity/v3/projects/234a9bec959e4e1681b28596dbe44adb/users/df7633b35b5e42dc9795c28765d17026/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:08.296699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.296990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.297269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.328843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] MFA Rules not processed for user `df7633b35b5e42dc9795c28765d17026`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:08.330125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:08.403460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1122/2226] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:08.416058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.417822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:08.478879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1506-e5a2-45ca-97a9-e9a4010bb870 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.482535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.483139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.483451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.483727 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.492543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.494008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.504459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1105/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:08.556571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:08.569014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0f628-42a4-493b-9180-8a6c441d3887 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:08.572890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:08.574243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:08.574612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:08.574944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:08.637288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:08.638735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7e29e-58fc-4e3e-9e79-dcec6f9ea989 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:08.648354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1123/2228] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:11.649366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:11.710890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:11.718869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c57e-49d3-4a95-80f0-6d4aa333394e service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:11.723935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:11.724568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:11.724853 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:11.725132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] PATH_INFO: `/v3/users/74620f7b96004504a40683303175e6b0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:11.726267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=74620f7b96004504a40683303175e6b0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:11.728266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:11.803691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74620f7b96004504a40683303175e6b0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:11.820444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74620f7b96004504a40683303175e6b0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:11.821221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74620f7b96004504a40683303175e6b0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:11.821861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74620f7b96004504a40683303175e6b0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:11.835957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7f2f95cf-6cdc-4d55-b128-265cb828f1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74620f7b96004504a40683303175e6b0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:11.866861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1106/2229] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:11 2022] DELETE /identity/v3/users/74620f7b96004504a40683303175e6b0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:11.881507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:11.882006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:11.882300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:11.909234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:11.910473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:11.927968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:11.958384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:11.958663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:11.958943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:11.980102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:11.981250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:11.986981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1124/2230] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:11.998454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:12.000114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:12.048578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1107/2231] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:19:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:12.067103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e0894-07a1-45e6-83ce-1ea905054ac9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:12.070782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:12.071427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:12.071708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:12.071982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:12.082957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:12.084481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:12.092387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:12.093910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:12.094724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1125/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:12.167360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf3abf-aef1-47fd-8784-c64386dcb982 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:12.173127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:12.173746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:12.174029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:12.174340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:12.183827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:12.185422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:12.193990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1108/2233] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 16:19:12.394988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:12.396489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:12.421866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45797236-3f7a-45fa-9010-423f9faf5ec6 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:12.430546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:12.431450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:12.432334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:12.432924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] PATH_INFO: `/v3/projects/e86c5a35cd7246c383df3dc2ede4c8ee` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:12.440399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86c5a35cd7246c383df3dc2ede4c8ee)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:12.446858 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:12.505896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86c5a35cd7246c383df3dc2ede4c8ee', 'request_id': 'req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.515379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86c5a35cd7246c383df3dc2ede4c8ee', 'request_id': 'req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.522533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86c5a35cd7246c383df3dc2ede4c8ee', 'request_id': 'req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.534620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] The token cache is being invalidate because project e86c5a35cd7246c383df3dc2ede4c8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:12.534917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.535717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d8b0f955-0bed-48bb-81f8-dcfa3238e9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.540559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1126/2234] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:12 2022] DELETE /identity/v3/projects/e86c5a35cd7246c383df3dc2ede4c8ee => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:12.541370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:12.552020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a71a34-7801-4b4a-b124-3dd48b2ea941 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:12.557361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:12.619006 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:12.619714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:12.620020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:12.620321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] PATH_INFO: `/v3/users/235694e694374d1d919d52369c04ac8e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:12.621483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235694e694374d1d919d52369c04ac8e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:12.623648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:12.680548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235694e694374d1d919d52369c04ac8e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.691198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235694e694374d1d919d52369c04ac8e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.692139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235694e694374d1d919d52369c04ac8e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.692663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235694e694374d1d919d52369c04ac8e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.702779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235694e694374d1d919d52369c04ac8e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:12.722140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1109/2235] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:12 2022] DELETE /identity/v3/users/235694e694374d1d919d52369c04ac8e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:12.813309 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:12.813597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:12.813945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:12.833678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:12.834901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:12.879779 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:12.907944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1127/2236] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:19:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:12.923329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:12.924793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:13.019644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57f30-3c72-49e1-8b0c-ed615a997879 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:13.025365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:13.025998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.026279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.027148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.036734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:13.054033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:13.055866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:13.056099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:13.074482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1110/2237] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:19:13.149670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238e27b-0acd-4e94-b513-b0b14038ee1b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:13.156703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:13.157397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.157680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.158010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] PATH_INFO: `/v3/projects/13564aa78a054e658cafe535ef3ff698` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.159490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13564aa78a054e658cafe535ef3ff698)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:13.161445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:13.215214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13564aa78a054e658cafe535ef3ff698', 'request_id': 'req-114ff297-f130-4ffb-9166-dbea1880596a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.223578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13564aa78a054e658cafe535ef3ff698', 'request_id': 'req-114ff297-f130-4ffb-9166-dbea1880596a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.230421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13564aa78a054e658cafe535ef3ff698', 'request_id': 'req-114ff297-f130-4ffb-9166-dbea1880596a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.240901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] The token cache is being invalidate because project 13564aa78a054e658cafe535ef3ff698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:13.241199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.241837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-114ff297-f130-4ffb-9166-dbea1880596a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.243349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1128/2238] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:13 2022] DELETE /identity/v3/projects/13564aa78a054e658cafe535ef3ff698 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:13.247426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:13.259429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9034b0-9db2-4a2a-8311-d761aac0812f service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:13.273583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:13.336506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:13.336981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.337283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.337642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] PATH_INFO: `/v3/users/df7633b35b5e42dc9795c28765d17026` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.338860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7633b35b5e42dc9795c28765d17026)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:13.340961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:13.341106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.341441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.341816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.362762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:13.364137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:13.393565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7633b35b5e42dc9795c28765d17026'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.403979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7633b35b5e42dc9795c28765d17026'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.405016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7633b35b5e42dc9795c28765d17026'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.405758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7633b35b5e42dc9795c28765d17026'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.416389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7633b35b5e42dc9795c28765d17026'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.428485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1129/2239] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:19:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:13.435535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1111/2240] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:13 2022] DELETE /identity/v3/users/df7633b35b5e42dc9795c28765d17026 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:13.444149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:13.446972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:13.479129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:13.529595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712597d-b872-4586-a6b4-e5ada2263749 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:13.596230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:13.596856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.597331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.597445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.660633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:13.662257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51dc7-bb9c-49ad-8496-b649cee3f2a9 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:13.670730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1130/2241] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:19:13.736018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:13.737714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:13.828303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2154-f864-4101-bc65-3a5e18a10294 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:13.833224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:13.833953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.834243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.834544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] PATH_INFO: `/v3/projects/234a9bec959e4e1681b28596dbe44adb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.835712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234a9bec959e4e1681b28596dbe44adb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:13.837612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:13.887698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:13.898515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234a9bec959e4e1681b28596dbe44adb', 'request_id': 'req-44d719ec-57af-4612-9064-6c6a9e2838b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.906031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234a9bec959e4e1681b28596dbe44adb', 'request_id': 'req-44d719ec-57af-4612-9064-6c6a9e2838b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.922630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234a9bec959e4e1681b28596dbe44adb', 'request_id': 'req-44d719ec-57af-4612-9064-6c6a9e2838b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.935010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] The token cache is being invalidate because project 234a9bec959e4e1681b28596dbe44adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:13.935010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.935472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-44d719ec-57af-4612-9064-6c6a9e2838b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:13.939258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1112/2242] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:13 2022] DELETE /identity/v3/projects/234a9bec959e4e1681b28596dbe44adb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:13.943967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925670-279f-4ec3-a9ef-de6f848e134d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:13.943967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925670-279f-4ec3-a9ef-de6f848e134d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:13.944647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925670-279f-4ec3-a9ef-de6f848e134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:13.965113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-36925670-279f-4ec3-a9ef-de6f848e134d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:13.966322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-36925670-279f-4ec3-a9ef-de6f848e134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.037906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1131/2243] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:14.046189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.046592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.047038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.064737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.065207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.065488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.069362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:14.070586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.079260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1113/2244] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:14.083514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.091453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:14.094262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.102174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:14.106989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2e315-928e-4e22-a4a9-d2e0eb3f9637 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.113070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.113830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.114211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.114821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.115624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:19:14.116027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.117721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.134173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1114/2245] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:19:14.139322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.153806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97535-c57b-4e31-b298-5942beb5f32a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.167747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.168362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.169328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.169607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.170275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.172283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.173235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1132/2246] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:19:14.191038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.193258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.203490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1115/2247] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:14 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:14.207424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.281500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c0a12-ac81-4e6f-86a0-e14be2f0b3f8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.287422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.288192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.288584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.289022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.289196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02de9-2e06-44c6-b592-99157c2f520b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.289969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.292909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.315760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1116/2248] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:14 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:14.319846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.331557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c328af-48eb-488f-b28e-e9b211057e79 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.342002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.342542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.342815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.343089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.343854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.345472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.359196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1117/2249] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:14.359523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.360146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.360282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.360429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.360706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.369928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dcce2-ad66-482c-ace9-48d5611ddf0f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.374787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.375307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.375607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.375880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.376768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.378498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.392592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1118/2250] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:14.393876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.404193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740879-7402-4943-bd07-4908eca3ddb6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.409074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.409721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.410001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.410272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] PATH_INFO: `/v3/projects/b0318dd06dc74631b3c6a502134e5bf8/users/bb5b0f4bc9624db5930c7d4f8667199e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.426484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0318dd06dc74631b3c6a502134e5bf8, user_id=bb5b0f4bc9624db5930c7d4f8667199e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.429356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.430745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.432406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534699ab-3a93-4ea9-8b61-25f677f56628 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.436781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:14.445273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1133/2251] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:19:14.446898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1119/2252] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:14 2022] PUT /identity/v3/projects/b0318dd06dc74631b3c6a502134e5bf8/users/bb5b0f4bc9624db5930c7d4f8667199e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:14.462724 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.463050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.463367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.498563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] MFA Rules not processed for user `bb5b0f4bc9624db5930c7d4f8667199e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:14.500800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.627717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1134/2253] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:19:14.638259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:14.646470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:14.688494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:14.724913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f909c4d-2226-4920-92ff-5c0f137c02d7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:14.733229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:14.733229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:14.733229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:14.733229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:14.741074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:14.742525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c48a3-fcbf-400f-8196-6b2a3bd4c886 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:14.752989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1120/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:16.786912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:16.788464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:16.865716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbcff5-176b-452c-98e9-6058a6bf2b0a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:16.870937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:16.871624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:16.871906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:16.872177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] PATH_INFO: `/v3/users/b62f9fe4e66e42e2b3d75bee462ec079` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:16.873733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b62f9fe4e66e42e2b3d75bee462ec079)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:16.876355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:16.923842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b62f9fe4e66e42e2b3d75bee462ec079'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:16.930681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b62f9fe4e66e42e2b3d75bee462ec079'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:16.931534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b62f9fe4e66e42e2b3d75bee462ec079'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:16.939599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b62f9fe4e66e42e2b3d75bee462ec079'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:16.951697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b62f9fe4e66e42e2b3d75bee462ec079'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:16.957534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1135/2255] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:16 2022] DELETE /identity/v3/users/b62f9fe4e66e42e2b3d75bee462ec079 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:16.962420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:16.962944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:16.963046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:16.982239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:16.983552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.044501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1121/2256] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:17.054405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.055861 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.066772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:17.123503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73122-ae72-4e3c-baff-89608b8dd88b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.127611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.128405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.128691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.128975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.139896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.141360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c6051-500d-40b1-9563-f2d162278fb5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.151194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1136/2257] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:17.326282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.328236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.350542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af76342-2a2f-458b-b355-bc5bc7576312 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.357307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.358198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.358320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.358584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] PATH_INFO: `/v3/projects/280dfa8a14794440aeac6c75a1c9d471` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.359749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] RBAC: Authorizing `identity:delete_project(project_id=280dfa8a14794440aeac6c75a1c9d471)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.361813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.428949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '280dfa8a14794440aeac6c75a1c9d471', 'request_id': 'req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:17.436627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '280dfa8a14794440aeac6c75a1c9d471', 'request_id': 'req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:17.443599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '280dfa8a14794440aeac6c75a1c9d471', 'request_id': 'req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:17.456656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] The token cache is being invalidate because project 280dfa8a14794440aeac6c75a1c9d471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:17.456942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:17.457560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9c66a3c1-77af-40bf-99e5-f94e7e2f095e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:17.459281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1122/2258] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:17 2022] DELETE /identity/v3/projects/280dfa8a14794440aeac6c75a1c9d471 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:17.464876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.465208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.465487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.487577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:17.489173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.562677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1137/2259] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:17.572252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.572707 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.573158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.595594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:17.596830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.605315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1123/2260] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:17.609600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.619640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f5d9-94ae-4b9e-8bda-3cfba7770931 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.625233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.625828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.626450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.626722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.627411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:19:17.627700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.629319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.641553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1138/2261] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:19:17.645972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.656193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32f135-4709-4bb8-b80b-8ac221b8cf75 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.661047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.661654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.661939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.662243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.662895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.664850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.697026 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1124/2262] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:17.698270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.708695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12a44-673d-4bbf-9a6c-8beec02de941 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.774645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.775272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.775557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.775831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.776480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.778438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.801886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1139/2263] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:17 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:19:17.806344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.818555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66d99-b303-4e31-b5dc-82761e9a4948 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.823425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.824050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.824329 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.824627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.825268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.826848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.839309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1125/2264] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:17.841920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.852289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfda09-19c6-4178-9c58-4bb9738b288a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.857647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.858068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.858351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.858626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.859314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.860917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17271b-d50c-4e59-a996-3a27a4d17da1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.873073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1140/2265] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:17.877242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:17.888360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4dd43-f9a5-4e17-85d1-6036d4f18670 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:17.893656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:17.897766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.897766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.897766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] PATH_INFO: `/v3/projects/568821f418bb465193aac9ec311b59b2/users/c7e2f2b39c84479aa194cfed89d8376b/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.913716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:17.921394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] RBAC: Authorizing `identity:create_grant(project_id=568821f418bb465193aac9ec311b59b2, user_id=c7e2f2b39c84479aa194cfed89d8376b, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:17.925222 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:17.940262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1126/2266] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:17 2022] PUT /identity/v3/projects/568821f418bb465193aac9ec311b59b2/users/c7e2f2b39c84479aa194cfed89d8376b/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:17.945431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b18397-7ee6-4e07-b169-1c98824732ac None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:17.945720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b18397-7ee6-4e07-b169-1c98824732ac None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:17.945994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b18397-7ee6-4e07-b169-1c98824732ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:17.965048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d2b18397-7ee6-4e07-b169-1c98824732ac None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:17.966298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d2b18397-7ee6-4e07-b169-1c98824732ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:17.975826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:18.043445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1141/2267] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:18.056475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:18.058265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:18.125893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504814-d116-449c-a99d-beb0d8de950a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:18.129998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:18.130675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:18.131039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:18.131581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:18.142290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:18.144226 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:18.157819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1127/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:22.570866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:22.571458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:22.571458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:22.608628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] MFA Rules not processed for user `c7e2f2b39c84479aa194cfed89d8376b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:22.609931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:22.696069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1142/2269] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:22.701076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:22.712827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd74e9-e429-4bed-a80e-630163e19214 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:22.718153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:22.718759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:22.719040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:22.719356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:22.719995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:22.721925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:22.756640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1128/2270] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:22 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:22.760806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:22.771984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74314-8fe8-414a-8923-7dfb85ff11f6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:22.897658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:22.898278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:22.898557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:22.898829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:22.899517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:22.902053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:22.926515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1143/2271] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:22 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:19:22.930733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:22.942134 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1b2a-88c6-4c45-b21c-02ffae458abf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:22.947058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:22.947921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:22.948920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:22.949287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:22.949938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:22.951951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:22.974477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1129/2272] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:22.978621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:22.990315 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e0d3a-6d4f-4aaa-8ae5-ebbda5cf9dc6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:22.998287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:22.999065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:22.999423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:22.999695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] PATH_INFO: `/v3/projects/2827b10c9af94b20a501bdb2add14e4d/users/93c60443e97c4a3c92b488afc86d28b7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:23.021332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2827b10c9af94b20a501bdb2add14e4d, user_id=93c60443e97c4a3c92b488afc86d28b7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:23.023870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:23.040953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1144/2273] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:22 2022] PUT /identity/v3/projects/2827b10c9af94b20a501bdb2add14e4d/users/93c60443e97c4a3c92b488afc86d28b7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:23.044791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:23.055775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249d86-19fe-467c-8005-283b86e09d1a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:23.128532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:23.129156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:23.129439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:23.129715 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:23.130369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:23.132572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49c60d-9c6f-4c82-a65e-5b32a4f789cd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:23.148525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1130/2274] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:23.149679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:23.161126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38208-0380-4f9d-8815-f9a2a24f1d2a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:23.166087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:23.166904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:23.167272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:23.167618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] PATH_INFO: `/v3/projects/2827b10c9af94b20a501bdb2add14e4d/users/93c60443e97c4a3c92b488afc86d28b7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:23.169702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2827b10c9af94b20a501bdb2add14e4d, user_id=93c60443e97c4a3c92b488afc86d28b7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:23.172518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:23.178822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:23.195330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1145/2275] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:23 2022] PUT /identity/v3/projects/2827b10c9af94b20a501bdb2add14e4d/users/93c60443e97c4a3c92b488afc86d28b7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:28.152879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f91ec-b14a-4d65-92cc-b0570dd397df None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:28.153471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f91ec-b14a-4d65-92cc-b0570dd397df None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:28.154170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f91ec-b14a-4d65-92cc-b0570dd397df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:28.198012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4e2f91ec-b14a-4d65-92cc-b0570dd397df None None] MFA Rules not processed for user `93c60443e97c4a3c92b488afc86d28b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:28.200495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4e2f91ec-b14a-4d65-92cc-b0570dd397df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:28.258676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:28.287256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1131/2276] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:28.299068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:28.300925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:28.426870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db3920-1b4f-4694-85fb-57d9b578e09b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:28.431389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:28.432359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:28.432748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:28.433643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:28.512563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:28.514177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:28.524252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1146/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:28.706632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e159aee-9487-40d9-9c96-3116fdb3a4bc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:28.707004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e159aee-9487-40d9-9c96-3116fdb3a4bc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:28.707467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e159aee-9487-40d9-9c96-3116fdb3a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:28.728548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5e159aee-9487-40d9-9c96-3116fdb3a4bc None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:28.729918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5e159aee-9487-40d9-9c96-3116fdb3a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:28.793385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1132/2278] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:19:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:28.806277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:28.808436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:28.889514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-a4d3-46b8-b03e-2b1373b4184a service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:28.895206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:28.895863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:28.896146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:28.896477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:28.905774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:28.907490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa7601-c0eb-4562-ab0d-ee32f7ac6dc5 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:28.917449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1147/2279] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:19:38.786169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=33 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:38.875094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:38.875430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:38.875691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:38.895972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] MFA Rules not processed for user `c7e2f2b39c84479aa194cfed89d8376b`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:38.897192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:38.904185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1133/2280] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:38.912970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:41.264690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960551-fff5-49b7-bd37-547dbbd5ca88 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:41.265360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960551-fff5-49b7-bd37-547dbbd5ca88 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:41.266018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960551-fff5-49b7-bd37-547dbbd5ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:41.301701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-34960551-fff5-49b7-bd37-547dbbd5ca88 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:41.303914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34960551-fff5-49b7-bd37-547dbbd5ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:41.319164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1148/2281] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:19:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:41.336200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:41.356743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5af70-afa9-4fa7-af61-b62cd0865395 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:41.375523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:41.376117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:41.376673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:41.376960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:41.387941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:41.389684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:41.408194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1134/2282] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:19:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:19:42.740267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d470c74-9cda-4950-81fd-b35b9c607c9a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:42.740814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d470c74-9cda-4950-81fd-b35b9c607c9a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:42.740814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d470c74-9cda-4950-81fd-b35b9c607c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:42.761891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4d470c74-9cda-4950-81fd-b35b9c607c9a None None] MFA Rules not processed for user `03eed7086183469f8cc1e765422bea87`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:42.763435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d470c74-9cda-4950-81fd-b35b9c607c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:42.831506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1149/2283] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:19:51.396135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=9 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:52.823416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=28 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:57.599418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:57.601082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:57.681204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7997ca-4126-489a-b75a-5a5d3f04d64e service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:57.686718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:57.687525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:57.687842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:57.688155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] PATH_INFO: `/v3/users/ce579ebd16f04855aca97038af95ec7f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:57.689452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce579ebd16f04855aca97038af95ec7f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:57.691452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:57.763798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce579ebd16f04855aca97038af95ec7f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:57.775686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce579ebd16f04855aca97038af95ec7f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:57.777078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce579ebd16f04855aca97038af95ec7f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:57.777798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce579ebd16f04855aca97038af95ec7f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:57.789858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce579ebd16f04855aca97038af95ec7f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:57.823307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1135/2284] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:19:57 2022] DELETE /identity/v3/users/ce579ebd16f04855aca97038af95ec7f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:19:57.826067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:57.826534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:57.826725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:57.852967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:57.854674 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:57.928296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1150/2285] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:57.940335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:57.941848 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:58.008926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499edbd3-c617-499a-8d37-05bc15bfca67 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:58.013492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:58.014210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:58.014491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:58.014766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:58.028002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:58.029739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ba1a-949b-4e08-adf4-b3900db7b8ad service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:58.044873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1136/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:19:58.554025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:58.555651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:58.571607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a381-03d8-44c5-81eb-9ffbed432781 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:58.585095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:58.585741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:58.586022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:58.586298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] PATH_INFO: `/v3/projects/a9bdb6983382496aa4d0363bec53e29d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:58.588348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9bdb6983382496aa4d0363bec53e29d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:58.590313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:58.672458 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9bdb6983382496aa4d0363bec53e29d', 'request_id': 'req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:58.682366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9bdb6983382496aa4d0363bec53e29d', 'request_id': 'req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:58.690469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9bdb6983382496aa4d0363bec53e29d', 'request_id': 'req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:58.703020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] The token cache is being invalidate because project a9bdb6983382496aa4d0363bec53e29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:19:58.703390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:58.704012 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5dc8c74e-c65c-4d81-b617-1fbc21d5f313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:19:58.705520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1151/2287] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:19:58 2022] DELETE /identity/v3/projects/a9bdb6983382496aa4d0363bec53e29d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:58.710476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:58.711572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:58.712116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:58.736057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:58.737933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:58.761549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:58.818171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1137/2288] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:58.830227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:58.830827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:58.830827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:58.875403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:58.877979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:58.888423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1152/2289] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:58.894440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:58.926752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94663fb7-d888-43b7-a375-a730387ce729 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:58.948394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:58.949175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:58.949513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:58.950861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:58.951604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:19:58.952321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:58.954115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:58.975447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1138/2290] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:19:58.979571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:58.992537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60e8be-3d27-4826-81e1-c79e1bee9752 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:58.998617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:58.999708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.000220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.000824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.002060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.004464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.048152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1153/2291] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:19:58 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:59.049868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:59.061829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6df26-50b5-4e31-a11a-a4ae8d1fe31c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:59.174901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:59.175666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.176018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.176342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.176997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.179244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.211523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1139/2292] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:19:59 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:19:59.216499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:59.229404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47697c-dbba-462f-a185-33a8b75101a1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:59.235505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:59.236171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.236528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.236857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.237586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.241384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.256594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1154/2293] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:19:59.257765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:59.271965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670ba52-eb6c-4a79-85d6-2876590bf402 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:59.278054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:59.278733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.279104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.279465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.280183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.281859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a972f44-737e-420a-97e0-8e021d26c4a5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.294233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1140/2294] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:19:59.298432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:59.316197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883c2d9-f25d-44e9-a604-1325707cb202 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:59.321191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:59.321905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.322831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.323311 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] PATH_INFO: `/v3/projects/6f37adec99d941df927b4adb80fc741d/users/913646543c8a40e782985c087997c4a4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.345345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f37adec99d941df927b4adb80fc741d, user_id=913646543c8a40e782985c087997c4a4, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.348505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.367617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1155/2295] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:19:59 2022] PUT /identity/v3/projects/6f37adec99d941df927b4adb80fc741d/users/913646543c8a40e782985c087997c4a4/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:19:59.379680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.380033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.380418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.413095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] MFA Rules not processed for user `913646543c8a40e782985c087997c4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:19:59.414335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:59.529766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1141/2296] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:19:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:19:59.540574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:19:59.543209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:19:59.567410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:19:59.618373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44548f-a667-4702-b3ef-84554e2d3724 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:19:59.625820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:19:59.626253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:19:59.626895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:19:59.627323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:19:59.637504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:19:59.639050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbbef8-3547-4c42-8f79-9440e7b67a9f service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:19:59.653614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1156/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:19:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:02.361464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:02.362992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:02.385509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:02.435718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7f20c-4e0b-4b01-ab23-5c1cdf02941d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:02.440670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:02.441396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:02.441753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:02.442104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] PATH_INFO: `/v3/users/baba9ef2b1234fb8a3fed5ae472f8195` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:02.443347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baba9ef2b1234fb8a3fed5ae472f8195)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:02.445345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:02.493935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baba9ef2b1234fb8a3fed5ae472f8195'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:02.502153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baba9ef2b1234fb8a3fed5ae472f8195'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:02.502919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baba9ef2b1234fb8a3fed5ae472f8195'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:02.503799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baba9ef2b1234fb8a3fed5ae472f8195'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:02.518197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baba9ef2b1234fb8a3fed5ae472f8195'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:02.556696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1142/2298] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:02 2022] DELETE /identity/v3/users/baba9ef2b1234fb8a3fed5ae472f8195 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:02.562849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:02.563406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:02.567355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:02.622768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:02.627391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:02.749699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1157/2299] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:02.756980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:02.758557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:02.832850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950cd55-0aa6-46ae-8b5a-5ef75e24ed42 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:02.840836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:02.841391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:02.841815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:02.842101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:02.853546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:02.855843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96694b18-5836-4a96-be85-fd0bfba48d02 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:02.867984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1143/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:03.259705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.263910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:03.289893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615119b-1610-4bbf-85c8-7686018c0d4c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.296261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.296890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.297165 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.297431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] PATH_INFO: `/v3/projects/0e1f4336492e410899d7c9f53af824fb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.298568 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1f4336492e410899d7c9f53af824fb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.300903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.351287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1f4336492e410899d7c9f53af824fb', 'request_id': 'req-ca32c251-380f-4025-b4f5-8c3a40727034'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:03.362940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1f4336492e410899d7c9f53af824fb', 'request_id': 'req-ca32c251-380f-4025-b4f5-8c3a40727034'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:03.370002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1f4336492e410899d7c9f53af824fb', 'request_id': 'req-ca32c251-380f-4025-b4f5-8c3a40727034'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:03.381897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] The token cache is being invalidate because project 0e1f4336492e410899d7c9f53af824fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:03.382196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:03.382883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ca32c251-380f-4025-b4f5-8c3a40727034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:03.384470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1158/2301] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:03 2022] DELETE /identity/v3/projects/0e1f4336492e410899d7c9f53af824fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:03.390059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.390570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.390759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.411636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:03.413454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:03.457684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:03.494194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1144/2302] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:20:03.502207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.502493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.502771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.524607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:03.525838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:03.534885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1159/2303] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:03.539241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.550568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19832927-2da1-45a6-8ca0-f73611a260ee None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.555547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.556391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.556391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.556537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.557169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:20:03.557475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.559673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.572563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1145/2304] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:20:03.577263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.586819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b3f6c-89dc-4702-8d24-5c6e13c08c2e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.591513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.592201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.592483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.592756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.593412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.595308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.626352 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1160/2305] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:20:03 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:20:03.630565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.643085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74134e08-1652-41b3-9fe9-bdea8a9281eb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.705873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.706480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.706796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.707070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.707750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.709652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.731363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1146/2306] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:20:03 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:20:03.735881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.748819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca75f1b-117d-4033-974c-2a7ee31b31e2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.755654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.756332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.756598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.757097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.758229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.759946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.770768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1161/2307] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:20:03.774821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.786106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ee4bc-5209-4e5f-b706-a1c02b439b62 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.791059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.791767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.792051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.792471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.793675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.795289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296cceb-080e-442e-a543-1eb6a6fdbdbe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.808462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1147/2308] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:20:03.812304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:03.832857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97dc74-7935-4038-aded-d628794d3ea0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:03.838005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:03.838625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.838924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.839752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] PATH_INFO: `/v3/projects/fd71f71c0df44862a8f740d77352d8bf/users/d2ff8b2d7a984a99816a81f75397cebc/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.842447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:03.863096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd71f71c0df44862a8f740d77352d8bf, user_id=d2ff8b2d7a984a99816a81f75397cebc, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:03.865892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:03.881242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1162/2309] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:20:03 2022] PUT /identity/v3/projects/fd71f71c0df44862a8f740d77352d8bf/users/d2ff8b2d7a984a99816a81f75397cebc/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:20:03.892131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:03.892423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:03.892704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:03.924491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] MFA Rules not processed for user `d2ff8b2d7a984a99816a81f75397cebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:03.926638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:03.994351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1148/2310] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:04.007003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:04.008776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:04.069605 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6737-6e87-47bf-afbb-7b90f443e4ef admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:04.073210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:04.073865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:04.074132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:04.074406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:04.082757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:04.084266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ff9d1-33e9-4149-82ef-7a96bc10c361 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:04.093590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1163/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:13.991482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:14.090174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:20.432892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:20.503554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8a8c7-6bea-4e5f-a0e7-804c617c6555 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:20.508399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:20.509117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:20.509520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:20.509876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] PATH_INFO: `/v3/users/d2ff8b2d7a984a99816a81f75397cebc` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:20.511144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ff8b2d7a984a99816a81f75397cebc)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:20.514259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:20.567595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ff8b2d7a984a99816a81f75397cebc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:20.575538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ff8b2d7a984a99816a81f75397cebc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:20.576266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ff8b2d7a984a99816a81f75397cebc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:20.577920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ff8b2d7a984a99816a81f75397cebc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:20.585440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ff8b2d7a984a99816a81f75397cebc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:20.606129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1149/2312] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:20 2022] DELETE /identity/v3/users/d2ff8b2d7a984a99816a81f75397cebc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:20.610978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:20.611497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:20.611591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:20.633279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:20.634579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:20.696147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1164/2313] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:20.706139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:20.707602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:20.776778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796e00-b72a-4f12-9c08-a742345277d6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:20.780773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:20.781387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:20.781675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:20.781956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:20.792258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:20.794330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf77af0-6bbd-4a70-befa-a79848a8ca3a service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:20.804666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1150/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:21.134382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.136018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:21.148504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01438-41c5-499f-85f4-68cd8cc85397 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.153792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.154504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.154786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.155075 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] PATH_INFO: `/v3/projects/fd71f71c0df44862a8f740d77352d8bf` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.156684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd71f71c0df44862a8f740d77352d8bf)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.158357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.205282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd71f71c0df44862a8f740d77352d8bf', 'request_id': 'req-07a8169b-8549-4bf2-9064-ef39ab8f611a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:21.213370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd71f71c0df44862a8f740d77352d8bf', 'request_id': 'req-07a8169b-8549-4bf2-9064-ef39ab8f611a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:21.220642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd71f71c0df44862a8f740d77352d8bf', 'request_id': 'req-07a8169b-8549-4bf2-9064-ef39ab8f611a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:21.231886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] The token cache is being invalidate because project fd71f71c0df44862a8f740d77352d8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:21.232182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:21.232844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07a8169b-8549-4bf2-9064-ef39ab8f611a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:21.234446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1165/2315] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:21 2022] DELETE /identity/v3/projects/fd71f71c0df44862a8f740d77352d8bf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:21.239380 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.239671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.239944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.260351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:21.261505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:21.280612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:21.324796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1151/2316] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:21.332382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.332671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.332938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.353832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:21.355235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:21.363480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1166/2317] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:21.367751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.377775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b62c-cdfc-4062-a6b8-a938c0999725 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.382341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.382931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.383209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.383511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.384147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:20:21.384448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.386021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.397946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1152/2318] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:20:21.402230 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.412493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebf297-da78-4e38-b75f-ddccea721c1e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.417260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.417857 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.418155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.418431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.419348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.421030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.457304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1167/2319] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:20:21 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:20:21.461796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.472668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01f670-c81e-4795-acbc-ed6971f9b862 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.532494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.533220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.533563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.533855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.534534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.536746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.561300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1153/2320] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:20:21 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:20:21.565702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.578709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efe24-a59f-4c27-a98a-49f63a1d4852 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.584735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.584735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.584735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.585182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.585841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.587493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.599280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1168/2321] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:20:21.602958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.613603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca5de-47ef-4f91-98aa-c82e50f49d74 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.619175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.619783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.620075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.620348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.621008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.628333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eae834-a6bb-4cc4-bf16-4b509e3c801b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.648734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1154/2322] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:20:21.653353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.665196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc972f-1e3d-45ec-ac06-1d8a5abcdc5b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.672431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.673153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.673438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.673718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] PATH_INFO: `/v3/projects/a9957d635d214a438bdcbe8901b926ef/users/e1331fa532f946039ac5dbbd78837fb1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.695761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9957d635d214a438bdcbe8901b926ef, user_id=e1331fa532f946039ac5dbbd78837fb1, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.698461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.717587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1169/2323] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:20:21 2022] PUT /identity/v3/projects/a9957d635d214a438bdcbe8901b926ef/users/e1331fa532f946039ac5dbbd78837fb1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:20:21.728328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.728642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.728994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.760232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] MFA Rules not processed for user `e1331fa532f946039ac5dbbd78837fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:21.761421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:21.829764 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1155/2324] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:21.841491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.843182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:21.866008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:21.903518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1679c1-7fdc-4401-8a2c-633cce0d5230 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:21.907445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:21.908057 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:21.908341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:21.908618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:21.917591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:21.919689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcd0bd-5c65-47dc-8981-baf7418bdeb5 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:21.928743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1170/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:21.954669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:21.974878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:22.028389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e995-a9d5-4cb3-b37f-84f5877d85ea None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:22.033404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:22.034162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:22.034450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:22.034729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] PATH_INFO: `/v3/users/e1331fa532f946039ac5dbbd78837fb1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:22.035941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1331fa532f946039ac5dbbd78837fb1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:22.038018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:22.093180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1331fa532f946039ac5dbbd78837fb1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.100448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1331fa532f946039ac5dbbd78837fb1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.101118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1331fa532f946039ac5dbbd78837fb1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.101735 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1331fa532f946039ac5dbbd78837fb1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.108780 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1331fa532f946039ac5dbbd78837fb1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.127304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1156/2326] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:21 2022] DELETE /identity/v3/users/e1331fa532f946039ac5dbbd78837fb1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:22.131526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:22.131820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:22.132102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:22.151661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:22.153523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:22.218491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1171/2327] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:22.225198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:22.226643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:22.298636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646ae06-e704-4d4f-a859-d6b2c425f461 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:22.302352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:22.302958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:22.303271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:22.303559 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:22.313069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:22.314492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:22.323900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1157/2328] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:22.599727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:22.601268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:22.614652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8c7c-bbc0-4419-84bf-876aa038547c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:22.619517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:22.620242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:22.620529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:22.620811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] PATH_INFO: `/v3/projects/a9957d635d214a438bdcbe8901b926ef` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:22.622211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9957d635d214a438bdcbe8901b926ef)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:22.631492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:22.679006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9957d635d214a438bdcbe8901b926ef', 'request_id': 'req-22d86830-beda-4344-aa86-662b9c4c6f60'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.686119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9957d635d214a438bdcbe8901b926ef', 'request_id': 'req-22d86830-beda-4344-aa86-662b9c4c6f60'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.692546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9957d635d214a438bdcbe8901b926ef', 'request_id': 'req-22d86830-beda-4344-aa86-662b9c4c6f60'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.703135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] The token cache is being invalidate because project a9957d635d214a438bdcbe8901b926ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:22.703456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.704112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:22.705636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1172/2329] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:22 2022] DELETE /identity/v3/projects/a9957d635d214a438bdcbe8901b926ef => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:27.750420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:27.752295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:27.802621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:27.846420 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27138ba-575c-4185-8c6c-f5c3354a942c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:27.923953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:27.924824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:27.925217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:27.925567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:27.928080 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:27.945426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:27.947092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b651616-5b0f-422b-9f26-f3a64492a18f service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:27.960159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1158/2330] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:29.497457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:29.499486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:29.552483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:29.570920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86830-beda-4344-aa86-662b9c4c6f60 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:29.575781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:29.576412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:29.576693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:29.576978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] PATH_INFO: `/v3/users/913646543c8a40e782985c087997c4a4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:29.578102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=913646543c8a40e782985c087997c4a4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:29.580119 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:29.628759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913646543c8a40e782985c087997c4a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:29.635942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913646543c8a40e782985c087997c4a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:29.637542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913646543c8a40e782985c087997c4a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:29.646950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913646543c8a40e782985c087997c4a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:29.660169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913646543c8a40e782985c087997c4a4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:29.666321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1173/2331] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:29 2022] DELETE /identity/v3/users/913646543c8a40e782985c087997c4a4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:29.670843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:29.671315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:29.671521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:29.693593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:29.694803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:29.760695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1159/2332] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:29.771971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:29.773465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:29.833499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493167d9-07a3-4ccb-ae6f-caf4292a4d52 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:29.837351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:29.837994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:29.838278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:29.838562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:29.850581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:29.852251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:29.862673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1174/2333] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:30.161824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:30.163320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:30.168717 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:30.170224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:30.178108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba9f6-81a2-4a9c-bf21-cd3b95d8d425 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:30.184685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:30.185349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:30.185634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:30.185912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] PATH_INFO: `/v3/users/bb5b0f4bc9624db5930c7d4f8667199e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:30.187073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5b0f4bc9624db5930c7d4f8667199e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:30.189189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:30.194814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e6022-a801-42bf-bad7-af21ff114dc4 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:30.200211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:30.201041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:30.201254 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:30.201591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] PATH_INFO: `/v3/projects/6f37adec99d941df927b4adb80fc741d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:30.202913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f37adec99d941df927b4adb80fc741d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:30.205725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:30.245913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5b0f4bc9624db5930c7d4f8667199e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.253579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5b0f4bc9624db5930c7d4f8667199e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.254247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5b0f4bc9624db5930c7d4f8667199e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.254869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5b0f4bc9624db5930c7d4f8667199e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.265746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ca8d7650-18fe-437e-87cc-75b2052c8cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5b0f4bc9624db5930c7d4f8667199e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.266731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f37adec99d941df927b4adb80fc741d', 'request_id': 'req-78498d76-aa94-43e3-806c-22f68ab7c42a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.271425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:30.274822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f37adec99d941df927b4adb80fc741d', 'request_id': 'req-78498d76-aa94-43e3-806c-22f68ab7c42a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.284205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f37adec99d941df927b4adb80fc741d', 'request_id': 'req-78498d76-aa94-43e3-806c-22f68ab7c42a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.286508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:30.291681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1160/2334] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:30 2022] DELETE /identity/v3/users/bb5b0f4bc9624db5930c7d4f8667199e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:30.295983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:30.296279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:30.296561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:30.298515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] The token cache is being invalidate because project 6f37adec99d941df927b4adb80fc741d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:30.299100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.300053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:30.301714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1175/2335] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:30 2022] DELETE /identity/v3/projects/6f37adec99d941df927b4adb80fc741d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:20:30.321518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:30.322813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:30.396586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1161/2336] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:30.407129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:30.408711 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:30.481620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498d76-aa94-43e3-806c-22f68ab7c42a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:30.533969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:30.535423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:30.535423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:30.535423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:30.551333 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:30.553475 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a55c2-6bc8-4721-a9bf-cec55d384dba service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:30.569326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1176/2337] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:20:30.950753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:30.952300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:30.969392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd8dd-5594-45c0-9d7c-1bca19ff1d9a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:30.974349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:30.975291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:30.975769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:30.976266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] PATH_INFO: `/v3/projects/b0318dd06dc74631b3c6a502134e5bf8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:30.977784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0318dd06dc74631b3c6a502134e5bf8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:30.979980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.075302 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0318dd06dc74631b3c6a502134e5bf8', 'request_id': 'req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:31.084267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0318dd06dc74631b3c6a502134e5bf8', 'request_id': 'req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:31.092838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0318dd06dc74631b3c6a502134e5bf8', 'request_id': 'req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:31.120036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] The token cache is being invalidate because project b0318dd06dc74631b3c6a502134e5bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:31.120361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:31.121225 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7fe955f0-f3c5-4907-9fd7-4a05fe2ee23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:31.123257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1162/2338] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:30 2022] DELETE /identity/v3/projects/b0318dd06dc74631b3c6a502134e5bf8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:31.128276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.128797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.129295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.175657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:31.176954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:31.261201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1177/2339] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:31.271894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.272314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.272658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.293655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:31.294887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:31.307961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1163/2340] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:31.312197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.322003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca4b4-6619-41af-bbb8-610d0211d918 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.327317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.327921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.328206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.328480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.329145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:20:31.329443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.331068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.348763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1178/2341] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:20:31.349876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.360531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61cbcc-51f3-42bb-9cbf-a787ff148941 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.371598 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.374109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.374109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.374109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.374800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.376558 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.416208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1164/2342] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:20:31 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:20:31.420240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.430768 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56041-6086-4a2a-9391-25237394f413 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.470132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:31.506017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.506707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.507096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.507499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.508338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.510561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.539301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1179/2343] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:20:31 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:20:31.540555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.554039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7af2b-54b2-462f-aa5c-48a098742bed admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.560133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.560751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.561035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.561342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.561965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.563622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.575388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1165/2344] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:20:31.579030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.589295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446f77-1f3e-4af6-afd1-9773abadb8a7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.595041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.595703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.596013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.596343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.596997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.598649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16663675-bea4-40a2-b521-167ddbe9675c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.610496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1180/2345] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:20:31.614597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.629935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aa10d-5746-4e56-a0c4-cc01b1a513ca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.634674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.635367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.635788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.636071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] PATH_INFO: `/v3/projects/4519a23f18c24f21aa88f58134ac765c/users/61152bb4942f480483ab97688e6e3c8d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.653600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4519a23f18c24f21aa88f58134ac765c, user_id=61152bb4942f480483ab97688e6e3c8d, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.656348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.669829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1166/2346] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:20:31 2022] PUT /identity/v3/projects/4519a23f18c24f21aa88f58134ac765c/users/61152bb4942f480483ab97688e6e3c8d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:20:31.680563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.680998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.681287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.712869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] MFA Rules not processed for user `61152bb4942f480483ab97688e6e3c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:31.714207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:31.791060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1181/2347] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:20:31.802359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:31.804099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:31.811166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:31.869103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f9333-8645-4c83-a520-ef863008951a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:31.873542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:31.874188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:31.874486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:31.874766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:31.885028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:31.886002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:31.898894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1167/2348] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:38.614841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:38.617637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:38.685164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee2b08-e6b2-4fc1-926a-7549b8815ab6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:38.689431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:38.690609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:38.690962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:38.691332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:38.693581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:38.770090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:38.773004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:38.774977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34538189-495f-47ae-a776-27733160cda2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:38.790132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1182/2349] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:41.891900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=24 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:42.264479 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:42.329794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11279476-051d-4231-84c5-0b65d5adabac service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:42.334325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:42.334952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:42.335251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:42.335532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] PATH_INFO: `/v3/users/61152bb4942f480483ab97688e6e3c8d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:42.336660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61152bb4942f480483ab97688e6e3c8d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:42.338630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:42.390077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61152bb4942f480483ab97688e6e3c8d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:42.397651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61152bb4942f480483ab97688e6e3c8d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:42.398395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61152bb4942f480483ab97688e6e3c8d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:42.399097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61152bb4942f480483ab97688e6e3c8d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:42.409183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61152bb4942f480483ab97688e6e3c8d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:42.428397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1168/2350] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:20:42 2022] DELETE /identity/v3/users/61152bb4942f480483ab97688e6e3c8d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:42.433837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:42.434417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:42.434417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:42.454351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:42.455655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:42.521096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1183/2351] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:42.531170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:42.532662 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:42.604199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464773f1-f89b-4a31-a85c-bb5b85761f3d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:42.608819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:42.609472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:42.609766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:42.610453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:42.627291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:42.627291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8deea3-1f50-4e7e-9296-1f5524677116 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:42.638135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1169/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:43.054928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.057152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:43.075607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2f4c8-fe47-489b-b1e3-afe6656227cc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.080304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.080947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.081251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.081882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] PATH_INFO: `/v3/projects/4519a23f18c24f21aa88f58134ac765c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.082888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4519a23f18c24f21aa88f58134ac765c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.084872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.143071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4519a23f18c24f21aa88f58134ac765c', 'request_id': 'req-43506b69-f03a-4233-8e11-f6c803b837b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:43.152799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4519a23f18c24f21aa88f58134ac765c', 'request_id': 'req-43506b69-f03a-4233-8e11-f6c803b837b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:43.160893 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4519a23f18c24f21aa88f58134ac765c', 'request_id': 'req-43506b69-f03a-4233-8e11-f6c803b837b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:43.178032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] The token cache is being invalidate because project 4519a23f18c24f21aa88f58134ac765c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:20:43.178336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:43.179014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43506b69-f03a-4233-8e11-f6c803b837b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:20:43.180847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1184/2353] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:20:43 2022] DELETE /identity/v3/projects/4519a23f18c24f21aa88f58134ac765c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:43.185539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.186059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.186160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.206947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:43.208144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:43.227577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:43.293493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1170/2354] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:43.295487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.295844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.296206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.319737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:43.321356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:43.331772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1185/2355] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:43.335801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.346325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c73b5-2388-46c7-8836-e42fee149ca0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.350967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.351612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.351899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.352172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.353090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:20:43.353403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.355068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.368963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1171/2356] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:20:43.373995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.384201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8087e2-2ffd-4df6-8304-18ff5879bcba None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.389780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.390374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.390652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.390923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.391662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.393626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.427644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1186/2357] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:20:43 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:20:43.430116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.442930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49cbc0-ad91-4233-a0de-a90a750485b5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.507650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.508348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.508640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.508919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.509550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.512152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.533759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1172/2358] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:20:43 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:20:43.537887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.548837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a995dc-84d2-4431-938c-181791b69ab4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.554117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.554753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.555042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.555424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.556077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.557691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.568244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1187/2359] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:20:43.572230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.583416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486b0e-c933-41ed-bd91-81e11ee08c2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.588136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.588677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.588961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.589242 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.589888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.591523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a82b-d5bc-4487-ad75-e86b5cf5d0f6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.602227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1173/2360] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:20:43.606470 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.616400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f858-5d68-441c-ace3-ad7f02d21028 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.621615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.622280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.622745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.623014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] PATH_INFO: `/v3/projects/80405e8d445d49199ed0d9ccd014dfb7/users/651091dd88274d2cb0b5b7afc0a18e5e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.639553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=80405e8d445d49199ed0d9ccd014dfb7, user_id=651091dd88274d2cb0b5b7afc0a18e5e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.642248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.656275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1188/2361] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:20:43 2022] PUT /identity/v3/projects/80405e8d445d49199ed0d9ccd014dfb7/users/651091dd88274d2cb0b5b7afc0a18e5e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:20:43.661717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.662007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.662283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.680987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:43.682198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:43.740091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1174/2362] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:43.750462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:43.752512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:43.771185 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:43.813266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123f7b8-8ab5-44bd-a948-b41c9b8848ae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:43.816964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:43.817570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:43.817859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:43.818134 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:43.827022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:43.828504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074e036-7ff0-449a-849b-2a5062429866 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:43.837845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1189/2363] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:44.381978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:44.393897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a9ab-4d8d-4da5-8259-5d4dea65778e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:44.399669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:44.400824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:44.403082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:44.403775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:44.407211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:44.439644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:44.489514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:44.491514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:44.506062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1175/2364] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:44.612317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ab88d-1151-4582-b1b7-caa29d1dc921 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:44.613236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ab88d-1151-4582-b1b7-caa29d1dc921 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:44.613728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ab88d-1151-4582-b1b7-caa29d1dc921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:44.649442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e16ab88d-1151-4582-b1b7-caa29d1dc921 None None] MFA Rules not processed for user `651091dd88274d2cb0b5b7afc0a18e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:44.650571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e16ab88d-1151-4582-b1b7-caa29d1dc921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:44.726234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1190/2365] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:44.737811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:20:44.739268 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:44.809446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9457c66-89dd-47b3-8667-07fdfa219868 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:20:44.814376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:20:44.814988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:44.815347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:44.815584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:44.825220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:20:44.826652 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fa078-0bcf-4999-9767-4cf242f7f0e1 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:20:44.836777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1176/2366] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:20:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:20:46.290248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:20:46.291140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:20:46.291885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:20:46.312823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] MFA Rules not processed for user `651091dd88274d2cb0b5b7afc0a18e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:20:46.314068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:20:46.321546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1191/2367] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:20:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:20:54.831751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=43 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:20:56.305419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=54 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:21:35.214616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f2ad-7a3b-47d3-97db-412199b1f9b7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:35.215247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f2ad-7a3b-47d3-97db-412199b1f9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:35.215247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f2ad-7a3b-47d3-97db-412199b1f9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:35.235487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9a32f2ad-7a3b-47d3-97db-412199b1f9b7 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:21:35.236838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9a32f2ad-7a3b-47d3-97db-412199b1f9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:35.296116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1177/2368] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:21:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:21:35.336836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:21:35.338256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:35.409395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683de6-94f0-4b3f-8a03-319bbf976801 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:21:35.414663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:21:35.415348 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:35.415633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:35.415912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:35.424744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:21:35.426341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47cfb8-6f64-470d-8c5a-f373d31d56ce service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:21:35.435098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1192/2369] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:21:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:21:45.287431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=25 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:21:45.431202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=32 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:21:55.434355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.434925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.434925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.436335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1178/2370] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:21:55.442805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05b33-dfa4-4789-a67b-e9267c0112cd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.443353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05b33-dfa4-4789-a67b-e9267c0112cd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.443451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05b33-dfa4-4789-a67b-e9267c0112cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.464152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6eb05b33-dfa4-4789-a67b-e9267c0112cd None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:21:55.465360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6eb05b33-dfa4-4789-a67b-e9267c0112cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:55.551744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1193/2371] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:21:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:21:55.563612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:21:55.565347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:55.628956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc78fe0-054d-4268-8e3b-2a23f2fa2fd3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:21:55.632745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f7dd7c12a4262a6ace2f07b6475c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:21:55.633357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.633643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.633920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.646297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:21:55.648288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80a776-154a-4a0c-a450-8c2e357ba158 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:21:55.659416 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1179/2372] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:21:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:21:55.729033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea582f2-b6b7-4d8d-9732-a9731a558326 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.729349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea582f2-b6b7-4d8d-9732-a9731a558326 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.729639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea582f2-b6b7-4d8d-9732-a9731a558326 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.731000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1194/2373] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:21:55.735829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebb9bc-beea-443f-8ff4-aa8c3aa9031c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.736124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebb9bc-beea-443f-8ff4-aa8c3aa9031c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.736404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebb9bc-beea-443f-8ff4-aa8c3aa9031c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.757357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d9ebb9bc-beea-443f-8ff4-aa8c3aa9031c None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:21:55.758562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9ebb9bc-beea-443f-8ff4-aa8c3aa9031c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:55.765788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1180/2374] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:21:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:21:55.777659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea582f2-b6b7-4d8d-9732-a9731a558326 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:21:55.788542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea582f2-b6b7-4d8d-9732-a9731a558326 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:21:55.792085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:21:55.792676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.792959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.793237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.800294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:21:55.801737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391f3f-9761-402d-8e6b-35ea8d1ecc46 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:21:55.809676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1195/2375] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:21:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:21:55.895771 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.896060 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.896343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.897815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1181/2376] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:21:55.903061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b2f7-e894-4d2f-8080-8567b6655525 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.903471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b2f7-e894-4d2f-8080-8567b6655525 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.903753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b2f7-e894-4d2f-8080-8567b6655525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.923365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9157b2f7-e894-4d2f-8080-8567b6655525 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:21:55.924579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9157b2f7-e894-4d2f-8080-8567b6655525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:55.933901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1196/2377] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:21:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:21:55.945034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:21:55.946732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:21:55.964889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa813e-dd14-47ab-a436-794ccd38512f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:21:55.968468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:21:55.969069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:21:55.969351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:21:55.969627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:21:55.979713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:21:55.981140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:21:55.991834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1182/2378] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:21:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:22:01.410351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:01.410856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:01.410964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:01.430771 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:22:01.432153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:01.439491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1197/2379] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:22:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:22:01.451918 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:01.466188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8710c-0fe7-4ef9-ab2d-ab6e9f802755 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:01.472831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:01.473426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:01.473705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:01.473978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:01.482103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:01.483725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:01.491981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1183/2380] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:22:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:22:04.177367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:04.187889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6788a-4ba9-4db7-b430-7b846d1b3333 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:04.191719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:04.192323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:04.192608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:04.192917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:04.194695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:04.212611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:04.214089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:04.228783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1198/2381] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:22:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:22:07.899509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:07.911772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b190-6ab4-473e-8a6c-211eb3d968cf service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:07.915687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:07.916424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:07.916898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:07.917186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:07.919083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:07.935782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:07.937227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895312f2-c50b-4eaa-8e87-5963ca1d42b9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:07.948830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1184/2382] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:22:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:22:14.220864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=49 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:22:15.098226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:15.100089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:15.114234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439b6b5-8846-44ce-a507-c1b54ad582da service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:15.119991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:15.120595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:15.120875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:15.121148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:15.122964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:15.211937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:15.213552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5efbe-b9ad-4c79-80e8-69666d1ec978 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:15.223250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1199/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:22:17.344020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:17.344602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:17.344705 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:17.363665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] MFA Rules not processed for user `651091dd88274d2cb0b5b7afc0a18e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:22:17.364851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:17.373788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1185/2384] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:22:25.218434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=28 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:22:27.358158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=50 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:22:35.485491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:35.486136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:35.486136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:35.511486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:22:35.512789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:35.520522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1200/2385] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:22:35.542812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:35.556369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7abd-76e5-4d1e-a07b-238549dd6cf1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:35.562016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:35.562803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:35.563719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:35.564103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:35.573531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:35.575250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:35.584481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1186/2386] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:22:45.505501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:22:45.581109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:22:53.528283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:22:53.537744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6887391-4284-4cdb-8f57-db7133f0e493 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:22:53.541351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:22:53.541945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:22:53.542225 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:22:53.542506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:22:53.544355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:22:53.554351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:22:53.555833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:22:53.565641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1201/2387] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:22:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:23:02.730015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:02.731831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:02.748504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe5c7e-5cb5-4042-bd95-ac5c8245a7c3 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:02.754723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:02.755513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:02.755801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:02.756047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:02.758015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:02.770675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:02.772319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:02.783642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1187/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:23:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:23:03.560608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:23:12.778544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:23:17.783503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:17.796748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733094c7-ebc4-433f-9d1c-de4612d46ee9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:17.802179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:17.802775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:17.803052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:17.803356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:17.805063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:17.862227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:17.863884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:17.872782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1202/2389] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:23:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:23:27.868639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=28 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:23:28.345589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:28.357693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9682-90df-473e-ac43-2a6e28d079d3 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:28.363975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:28.364643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:28.364965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:28.365277 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:28.367153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:28.429169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:28.430911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:28.440905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1188/2390] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:23:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:23:30.384223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:30.386465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:30.446372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:30.460630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1478-b996-4417-9900-40adecd45b59 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:30.464101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1209e-e4d2-478c-b115-c09ee1bf4e8a service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:30.465647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:30.466362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:30.466778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:30.467097 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] PATH_INFO: `/v3/users/651091dd88274d2cb0b5b7afc0a18e5e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:30.468447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=651091dd88274d2cb0b5b7afc0a18e5e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:30.469296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:30.469896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:30.470180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:30.470461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:30.470597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:30.472259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:30.484365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:30.486222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:30.496676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1203/2391] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:23:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:23:30.523883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651091dd88274d2cb0b5b7afc0a18e5e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.532013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651091dd88274d2cb0b5b7afc0a18e5e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.532721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651091dd88274d2cb0b5b7afc0a18e5e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.533555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651091dd88274d2cb0b5b7afc0a18e5e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.541023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-913cdc63-f5ef-4a88-a16a-e105254de4bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651091dd88274d2cb0b5b7afc0a18e5e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.559095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1189/2392] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:23:30 2022] DELETE /identity/v3/users/651091dd88274d2cb0b5b7afc0a18e5e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:23:30.733399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:30.745013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae174f8-d6fb-4fb4-b61c-6f610a71335d service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:30.812778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:30.813324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:30.813610 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:30.813890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] PATH_INFO: `/v3/projects/80405e8d445d49199ed0d9ccd014dfb7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:30.815022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80405e8d445d49199ed0d9ccd014dfb7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:30.816992 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:30.862881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80405e8d445d49199ed0d9ccd014dfb7', 'request_id': 'req-d2e79124-d8a4-423e-8367-91d575e55aa4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.870493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80405e8d445d49199ed0d9ccd014dfb7', 'request_id': 'req-d2e79124-d8a4-423e-8367-91d575e55aa4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.876762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80405e8d445d49199ed0d9ccd014dfb7', 'request_id': 'req-d2e79124-d8a4-423e-8367-91d575e55aa4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.888255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] The token cache is being invalidate because project 80405e8d445d49199ed0d9ccd014dfb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:23:30.888547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.889211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:23:30.891193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1204/2393] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:23:30 2022] DELETE /identity/v3/projects/80405e8d445d49199ed0d9ccd014dfb7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:23:40.555255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:23:40.884972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:23:47.988285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598d59-2c01-4f29-83b0-478cec2f2b2d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:47.988798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598d59-2c01-4f29-83b0-478cec2f2b2d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:47.988905 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598d59-2c01-4f29-83b0-478cec2f2b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:48.009800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8b598d59-2c01-4f29-83b0-478cec2f2b2d None None] MFA Rules not processed for user `03eed7086183469f8cc1e765422bea87`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:23:48.011011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8b598d59-2c01-4f29-83b0-478cec2f2b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:48.072512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1190/2394] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:23:54.364002 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:23:54.365591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:54.448452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79124-d8a4-423e-8367-91d575e55aa4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:23:54.454879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:23:54.455588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:23:54.455912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:23:54.456192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:23:54.458054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:23:54.471527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:23:54.473154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:23:54.485229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1205/2395] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:23:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:23:58.064383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=25 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:04.480212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=34 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:07.597657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:07.598165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:07.598281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:07.617733 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] MFA Rules not processed for user `c3dc3a92c6424cdbb3a2d3c1baaa95e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:24:07.618950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:07.675354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1191/2396] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:24:07.981219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:07.982623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:08.041661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336492b-3cba-4859-a33c-719bdec541d9 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:08.045346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:08.045949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:08.046235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:08.046512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:08.048289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:08.057785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:08.059212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:08.067795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1206/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:08.088266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:08.102562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375e0f5-c1eb-49bc-a040-42242b02202f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:08.111724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:08.111724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:08.111724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:08.111724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:08.113573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:08.179664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:08.181263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:08.191152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1192/2398] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:08.403734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:08.405541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:08.502991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544441-717f-49c0-a332-52f27ad39f6d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:08.509545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:08.510054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:08.510343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:08.510618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:08.512441 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:08.524933 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:08.526527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e20926-4829-4817-ae20-f17e48427392 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:08.537164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1207/2399] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:16.881709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:16.884752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:16.900011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32e545-b293-4489-aa9e-224d53adab93 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:16.905648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:16.906255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:16.906538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:16.907327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:16.909066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:16.921591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:16.923799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:16.934640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1193/2400] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:18.532300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=58 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:26.609163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:26.610483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:26.610483 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:26.633194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:24:26.634409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:26.703077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1208/2401] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:24:26.777566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:26.780019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:26.876786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7608d4-68d0-4602-a4e8-4ba36fd23d9a service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:26.883710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:26.884957 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:26.885663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:26.886216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:26.900882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:26.902514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9792eb-eb88-417d-9fd0-cebf79c9adee service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:26.914419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1194/2402] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:24:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:24:27.274181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:27.275645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:27.287139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4553c-13bf-4cef-aaa6-60ee59a3cef3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:27.290730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:27.291358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:27.291643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:27.291918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:27.293681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:27.361079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:27.362607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3222f-cd61-41b7-a685-7254c345d41c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:27.373865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1209/2403] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:28.265020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013952e0-e298-4841-b422-4764e96b208e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.265638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013952e0-e298-4841-b422-4764e96b208e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.265758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013952e0-e298-4841-b422-4764e96b208e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.267390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1195/2404] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:24:28.278872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c3df-7704-475a-973f-06ba402b1d8e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.279472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c3df-7704-475a-973f-06ba402b1d8e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.279472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c3df-7704-475a-973f-06ba402b1d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.301288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-80e4c3df-7704-475a-973f-06ba402b1d8e None None] MFA Rules not processed for user `393f32c7fcdf49219d155bb22fd3d048`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:24:28.302518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-80e4c3df-7704-475a-973f-06ba402b1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:28.309136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1210/2405] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:24:28.321267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013952e0-e298-4841-b422-4764e96b208e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:28.327463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:28.334574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013952e0-e298-4841-b422-4764e96b208e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:28.338138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:28.338755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.339048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.339378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.348086 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:28.349537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c59b9-cbb3-4d91-be68-7516e2dfa2a6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:28.358929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1196/2406] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:28.445004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255721-3a8c-4337-bbd5-c42c83914909 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.445667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255721-3a8c-4337-bbd5-c42c83914909 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.445974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255721-3a8c-4337-bbd5-c42c83914909 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.447346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1211/2407] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 30 16:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 16:24:28.453163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.453454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.453728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.474341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] MFA Rules not processed for user `5ce87b2207504e839bcd544faa3aa7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:24:28.475545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:28.485953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1197/2408] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 30 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:24:28.496827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73255721-3a8c-4337-bbd5-c42c83914909 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:28.508035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73255721-3a8c-4337-bbd5-c42c83914909 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:28.511765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:28.512366 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:28.512650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:28.512924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:28.524018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:28.525476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:28.537188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1212/2409] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:24:34.601029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:34.613630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26af98-17e9-45ff-aa8d-b5c1847325d6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:34.628967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:34.629713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:34.630273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:34.630636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:34.632719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:34.649768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:34.651446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:34.663574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1198/2410] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:24:34.692487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:34.694913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:34.711006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fef6-e0f4-41c3-b089-8ff1ca38879a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:34.716206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:34.716722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:34.717006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:34.717284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] PATH_INFO: `/v3/users/c7e2f2b39c84479aa194cfed89d8376b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:34.718438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e2f2b39c84479aa194cfed89d8376b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:34.721103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:34.770564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e2f2b39c84479aa194cfed89d8376b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:34.777995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e2f2b39c84479aa194cfed89d8376b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:34.778732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e2f2b39c84479aa194cfed89d8376b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:34.790487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e2f2b39c84479aa194cfed89d8376b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:34.800236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:34.806439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e2f2b39c84479aa194cfed89d8376b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:34.814159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1213/2411] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:24:34 2022] DELETE /identity/v3/users/c7e2f2b39c84479aa194cfed89d8376b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:24:34.987866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:35.004444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac870bf1-ffc7-4f02-99c9-0f241ce81583 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:35.069997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:35.070660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:35.070943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:35.071249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] PATH_INFO: `/v3/projects/568821f418bb465193aac9ec311b59b2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:35.072373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568821f418bb465193aac9ec311b59b2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:35.074270 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:35.120027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568821f418bb465193aac9ec311b59b2', 'request_id': 'req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.126765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568821f418bb465193aac9ec311b59b2', 'request_id': 'req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.132588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568821f418bb465193aac9ec311b59b2', 'request_id': 'req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.145243 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] The token cache is being invalidate because project 568821f418bb465193aac9ec311b59b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:24:35.145536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.146175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.147716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1199/2412] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:24:34 2022] DELETE /identity/v3/projects/568821f418bb465193aac9ec311b59b2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:24:35.151884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:35.153286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:35.167077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966acd-4eb7-4fb8-8b23-b5a039cd4899 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:35.229924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:35.230562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:35.230849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:35.231123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] PATH_INFO: `/v3/users/93c60443e97c4a3c92b488afc86d28b7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:35.232306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c60443e97c4a3c92b488afc86d28b7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:35.234258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:35.282080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c60443e97c4a3c92b488afc86d28b7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.289026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c60443e97c4a3c92b488afc86d28b7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.289704 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c60443e97c4a3c92b488afc86d28b7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.299008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c60443e97c4a3c92b488afc86d28b7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.310985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35c1238a-3a73-4556-9a47-bb380fe77ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c60443e97c4a3c92b488afc86d28b7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.318120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1214/2413] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:24:35 2022] DELETE /identity/v3/users/93c60443e97c4a3c92b488afc86d28b7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:24:35.502189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:35.503930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:35.519062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd45-68d8-4ba1-a82e-62801f3b7b79 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:35.578039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:35.579087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:35.579087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:35.579347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] PATH_INFO: `/v3/projects/2827b10c9af94b20a501bdb2add14e4d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:35.580348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2827b10c9af94b20a501bdb2add14e4d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:35.582241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:35.583683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:35.631311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2827b10c9af94b20a501bdb2add14e4d', 'request_id': 'req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.637694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2827b10c9af94b20a501bdb2add14e4d', 'request_id': 'req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.648469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2827b10c9af94b20a501bdb2add14e4d', 'request_id': 'req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.663369 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] The token cache is being invalidate because project 2827b10c9af94b20a501bdb2add14e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:24:35.663675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.664360 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:24:35.665931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1200/2414] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:24:35 2022] DELETE /identity/v3/projects/2827b10c9af94b20a501bdb2add14e4d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:24:45.314449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:45.660353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:24:58.374859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:58.375466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:58.375466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:58.398843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] MFA Rules not processed for user `ecf0e8a3fae04b499310fe0d35c08269`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:24:58.400082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:58.458817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1215/2415] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:24:58.796517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:24:58.798137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:58.863482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db7d0d-00c6-4dd9-a449-0b3bd84f437f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:24:58.867161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:24:58.867868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:24:58.868189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:24:58.868465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:24:58.870207 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:24:58.888650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:24:58.890109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:24:58.898674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1201/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:25:08.451647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:08.895028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=26 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:20.104820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:20.106249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:20.147989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabbaf9-f375-4ec4-9569-8ee8dcbd5eaa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:20.151705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:20.152306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:20.152582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:20.152849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:20.154566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:20.260392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:20.261846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:20.281507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1216/2417] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:25:23.966875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:23.968345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:23.996574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c200-c77c-48a7-9e2d-c200ff5c6ef9 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.001295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.002534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.002819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.003096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] PATH_INFO: `/v3/users/03eed7086183469f8cc1e765422bea87` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.004605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03eed7086183469f8cc1e765422bea87)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.006534 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.054602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03eed7086183469f8cc1e765422bea87'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.061509 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03eed7086183469f8cc1e765422bea87'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.062184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03eed7086183469f8cc1e765422bea87'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.062772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03eed7086183469f8cc1e765422bea87'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.071069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ba3edc58-cb03-4001-9f51-90f26e5e1dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03eed7086183469f8cc1e765422bea87'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.088520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1202/2418] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:25:23 2022] DELETE /identity/v3/users/03eed7086183469f8cc1e765422bea87 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:25:24.269328 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.279770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813fd4-3032-4884-b823-a5e6666bee8a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.337601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.338235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.338520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.338819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] PATH_INFO: `/v3/projects/dff1326e1026492080d292aa3a157366` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.339996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff1326e1026492080d292aa3a157366)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.341921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.365509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:24.393435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff1326e1026492080d292aa3a157366', 'request_id': 'req-131a9f9b-414e-4b29-be94-062d4f0414e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.400913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff1326e1026492080d292aa3a157366', 'request_id': 'req-131a9f9b-414e-4b29-be94-062d4f0414e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.407774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff1326e1026492080d292aa3a157366', 'request_id': 'req-131a9f9b-414e-4b29-be94-062d4f0414e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.418780 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] The token cache is being invalidate because project dff1326e1026492080d292aa3a157366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:25:24.419097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.419770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-131a9f9b-414e-4b29-be94-062d4f0414e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:25:24.421268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1217/2419] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:25:24 2022] DELETE /identity/v3/projects/dff1326e1026492080d292aa3a157366 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:25:24.426941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.427324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.427610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.485116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:24.486327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:24.545257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1203/2420] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:25:24.553824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.554113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.554393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.575067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:24.576317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:24.584920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1218/2421] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:25:24.589152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.598854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f570e3-2af1-41be-a115-a3e0f4377fc1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.603716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.604317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.604603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.604882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.605532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:25:24.605842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.607491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.619241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1204/2422] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:25:24.623522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.633595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c06c-bb40-4f9a-9739-d497d846b761 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.638251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.638841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.639120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.639427 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.640059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.642415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.670249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1219/2423] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:25:24 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:25:24.674885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.684798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf03b-84c5-459e-847b-68d8d41a6033 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.696636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:24.742475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.743076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.743395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.743673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.744309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.746227 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.767437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1205/2424] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:25:24 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:25:24.771676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.782043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531de7-87a5-45ea-bb91-dc76fdc981ba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.786787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.787425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.787708 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.787982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.788628 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.790402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.801892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1220/2425] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:25:24.805947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.818013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ba54-10ce-4a38-acaa-4bd066b92900 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.822669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.823296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.823585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.823864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.824508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.826109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b26e9-a152-4ec9-a295-232959839ff3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.837178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1206/2426] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:25:24.843115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.853570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbf01f-cf2c-4eb6-824e-9233e2453197 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:24.858590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:24.859257 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.859543 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.859817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] PATH_INFO: `/v3/projects/d08b7bad519b4188b8cef19735f023fe/users/ea15538623ef4bacb42c4e03e2db72c5/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.874778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08b7bad519b4188b8cef19735f023fe, user_id=ea15538623ef4bacb42c4e03e2db72c5, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:24.878069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:24.894673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1221/2427] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:25:24 2022] PUT /identity/v3/projects/d08b7bad519b4188b8cef19735f023fe/users/ea15538623ef4bacb42c4e03e2db72c5/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:25:24.900090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1749fc-e564-48b4-8cd9-8f14846acf88 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:24.900326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1749fc-e564-48b4-8cd9-8f14846acf88 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:24.900609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1749fc-e564-48b4-8cd9-8f14846acf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:24.919445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6d1749fc-e564-48b4-8cd9-8f14846acf88 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:24.920648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6d1749fc-e564-48b4-8cd9-8f14846acf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:24.980146 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1207/2428] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:25:24.990276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:24.991967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:25.055056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8be5-735d-49c2-8ad1-8b981810c08e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:25.058729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:25.059367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:25.059653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:25.059930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:25.069051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:25.070491 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:25.080067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1222/2429] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:25:29.070113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd618-45d5-4362-b654-9f93143ff70b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:29.070654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd618-45d5-4362-b654-9f93143ff70b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:29.070830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd618-45d5-4362-b654-9f93143ff70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:29.100575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8d6cd618-45d5-4362-b654-9f93143ff70b None None] MFA Rules not processed for user `ea15538623ef4bacb42c4e03e2db72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:29.101832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8d6cd618-45d5-4362-b654-9f93143ff70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:29.172301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1208/2430] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:25:29.191445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:29.193188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:29.262693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:29.326021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d958-1a57-44c3-a89a-3df1847f1e06 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:29.333369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:29.334135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:29.334503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:29.334854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:29.344170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:29.345935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:29.355766 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1223/2431] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:25:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:25:30.774767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2e49-9ac4-4d36-ae11-68a7fc07eb32 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:30.775275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2e49-9ac4-4d36-ae11-68a7fc07eb32 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:30.775465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2e49-9ac4-4d36-ae11-68a7fc07eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:30.796036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-736a2e49-9ac4-4d36-ae11-68a7fc07eb32 None None] MFA Rules not processed for user `ea15538623ef4bacb42c4e03e2db72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:30.797228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-736a2e49-9ac4-4d36-ae11-68a7fc07eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:30.804448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1209/2432] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:25:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:25:31.997166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:31.998686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:32.097800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a655f5c-96fa-470a-b418-63f2fa8cb53e service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:32.103168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:32.103834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:32.104121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:32.104398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:32.106304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:32.119681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:32.121319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:32.132986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1224/2433] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:25:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:25:40.790407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:42.127760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=66 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:25:58.532807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccffa-8998-4928-ad15-72338228bb13 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:58.533333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccffa-8998-4928-ad15-72338228bb13 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:58.533431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccffa-8998-4928-ad15-72338228bb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:58.565412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dc3ccffa-8998-4928-ad15-72338228bb13 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:25:58.566632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dc3ccffa-8998-4928-ad15-72338228bb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:58.623023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1210/2434] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:25:58.669842 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:25:58.671275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:25:58.739912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f217-0389-455f-837e-23d78c81b664 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:25:58.745150 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:25:58.745748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:25:58.746027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:25:58.746298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:25:58.754518 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:25:58.756145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:25:58.764107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1225/2435] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:25:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:26:08.616507 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=29 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:08.760889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=32 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:15.298084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:15.298585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:15.298679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:15.318841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:26:15.320167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:15.327260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1211/2436] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:26:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:26:15.338924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:15.349837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62591fd5-d8d3-4500-b0fb-dfe82912a8b6 service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:15.354827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:15.355446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:15.355735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:15.356005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:15.364751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:15.366382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d361dd-89dd-4598-8092-2ccae48d71fc service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:15.375043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1226/2437] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:26:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:26:25.313399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:25.371473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:34.197814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:34.216954 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed8c0e-64a0-43c1-b9cc-1a2a496170a9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:34.220578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:34.221172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:34.221452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:34.221726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:34.223468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:34.235985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:34.237436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:34.246903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1212/2438] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:26:35.466587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:35.467077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:35.467292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:35.487437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:26:35.488687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:35.496917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1227/2439] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:26:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:26:35.510022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:35.521130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd391c-848b-4944-9e54-58ec575f7e5f service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:35.526583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:35.527186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:35.527500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:35.527779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:35.536345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:35.538468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:35.549445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1213/2440] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:26:40.672231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:40.673889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:40.688270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989cf66-00f7-4be7-ae46-481af0dc4d80 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:40.693517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:40.694118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:40.694394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:40.694662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] PATH_INFO: `/v3/users/af14713dfbb348fa8824570570225de4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:40.695985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=af14713dfbb348fa8824570570225de4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:40.699349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:40.757126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af14713dfbb348fa8824570570225de4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:40.764464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af14713dfbb348fa8824570570225de4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:40.765175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af14713dfbb348fa8824570570225de4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:40.774396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af14713dfbb348fa8824570570225de4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:40.785823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af14713dfbb348fa8824570570225de4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:40.791926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1228/2441] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:26:40 2022] DELETE /identity/v3/users/af14713dfbb348fa8824570570225de4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:26:40.971364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:40.992477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d82a56-f57a-49ca-b8a4-de08fd65f8bc service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:41.056442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:41.057075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:41.057363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:41.057639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] PATH_INFO: `/v3/projects/43a1fa996402448091ef05ed5b8467af` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:41.058773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a1fa996402448091ef05ed5b8467af)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:41.060893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:41.107820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a1fa996402448091ef05ed5b8467af', 'request_id': 'req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.114052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a1fa996402448091ef05ed5b8467af', 'request_id': 'req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.119739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a1fa996402448091ef05ed5b8467af', 'request_id': 'req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.132333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] The token cache is being invalidate because project 43a1fa996402448091ef05ed5b8467af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:26:41.132628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.133376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.134739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1214/2442] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:26:40 2022] DELETE /identity/v3/projects/43a1fa996402448091ef05ed5b8467af => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:26:41.138754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:41.140454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:41.154229 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce980-b8a8-4305-8cd2-aa287da10fca admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:41.212361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:41.212998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:41.213284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:41.213561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] PATH_INFO: `/v3/users/ef855b112268459e899e30da4f5d2cf7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:41.214712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef855b112268459e899e30da4f5d2cf7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:41.216683 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:41.263617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef855b112268459e899e30da4f5d2cf7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.270486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef855b112268459e899e30da4f5d2cf7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.271705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef855b112268459e899e30da4f5d2cf7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.279533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef855b112268459e899e30da4f5d2cf7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.284553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:41.297729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef855b112268459e899e30da4f5d2cf7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.305733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1229/2443] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:26:41 2022] DELETE /identity/v3/users/ef855b112268459e899e30da4f5d2cf7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:26:41.486271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:26:41.487760 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:26:41.500785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03eb5e-34e8-48a5-8ee1-50c92d45416f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:26:41.554190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:41.562046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:26:41.563114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:26:41.563395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:26:41.563795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] PATH_INFO: `/v3/projects/23ca2d1ee84e4a959766d2954348a778` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:26:41.565231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ca2d1ee84e4a959766d2954348a778)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:26:41.567474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:26:41.616206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ca2d1ee84e4a959766d2954348a778', 'request_id': 'req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.622950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ca2d1ee84e4a959766d2954348a778', 'request_id': 'req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.628704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ca2d1ee84e4a959766d2954348a778', 'request_id': 'req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.643305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] The token cache is being invalidate because project 23ca2d1ee84e4a959766d2954348a778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:26:41.643602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.644147 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:26:41.645668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1215/2444] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:26:41 2022] DELETE /identity/v3/projects/23ca2d1ee84e4a959766d2954348a778 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:26:51.300586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:26:51.639887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:04.336850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:04.338425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:04.425371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2cc84-08de-410c-bc2f-ad1d780711ee admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:04.503750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:04.505636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:04.506332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:04.506332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:04.513397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:04.613653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:04.615120 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:04.634856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1230/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:09.930311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:09.931786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:09.944041 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6e8e2-5679-4e49-a663-823a1fb4ad51 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:09.947597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:09.948186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:09.948465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:09.948739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:09.950480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:10.013883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:10.015351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:10.025051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1216/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:13.264938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:13.266433 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:13.283167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84bbb5-cbeb-4009-9fc4-a4c70c6cbe80 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:13.288602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:13.289467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.289706 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.289981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] PATH_INFO: `/v3/users/ecf0e8a3fae04b499310fe0d35c08269` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.306932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf0e8a3fae04b499310fe0d35c08269)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:13.308960 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:13.344813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:13.360626 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf0e8a3fae04b499310fe0d35c08269'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.368989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf0e8a3fae04b499310fe0d35c08269'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.369812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf0e8a3fae04b499310fe0d35c08269'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.377856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf0e8a3fae04b499310fe0d35c08269'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.395938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1c445a14-6372-4400-9089-334684e69af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf0e8a3fae04b499310fe0d35c08269'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.401973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1231/2447] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:13 2022] DELETE /identity/v3/users/ecf0e8a3fae04b499310fe0d35c08269 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:13.590173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:13.602448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa47b0-6a79-4aab-9a55-aa56acb7461a service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:13.691205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:13.691880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.692166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.692445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] PATH_INFO: `/v3/projects/1c0d98b877d6458cb0e4c49dc8a57485` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.703487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0d98b877d6458cb0e4c49dc8a57485)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:13.705426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:13.757092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0d98b877d6458cb0e4c49dc8a57485', 'request_id': 'req-5f038c90-6102-452d-be47-a58e3720b7fa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.764431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0d98b877d6458cb0e4c49dc8a57485', 'request_id': 'req-5f038c90-6102-452d-be47-a58e3720b7fa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.770730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0d98b877d6458cb0e4c49dc8a57485', 'request_id': 'req-5f038c90-6102-452d-be47-a58e3720b7fa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.786855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] The token cache is being invalidate because project 1c0d98b877d6458cb0e4c49dc8a57485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:13.787149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.787808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5f038c90-6102-452d-be47-a58e3720b7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:13.789830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1217/2448] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:13 2022] DELETE /identity/v3/projects/1c0d98b877d6458cb0e4c49dc8a57485 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:13.795209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.795577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.795882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.816396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:13.817616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:13.878029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1232/2449] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:27:13.889327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.889628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.890079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.911628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:13.912864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:13.922518 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1218/2450] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:13.926929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:13.937721 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7564e-7c22-4d6c-ad71-76e6de057e8e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:13.942334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:13.942930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.943211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.943517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.944163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:13.944467 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:13.946061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:13.957677 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1233/2451] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:13.962738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:13.972359 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d276880-db54-42fd-87b0-56e90beca73a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:13.977903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:13.978590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:13.978881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:13.979173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:13.979840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:13.981758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:13.987038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:14.021675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1219/2452] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:13 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:14.026332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.036335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d7d2-44c5-4869-aa3b-bea11a4c5e73 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.092358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.092884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.093169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.093447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.094099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.096124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.120978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1234/2453] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:14 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:14.125217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.135974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3eb23-ae5e-4883-a09c-f61c6df99b56 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.140671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.141263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.141547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.141822 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.142487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.144117 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.154487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1220/2454] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:14.158665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.169454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6c92-6b31-41bb-818e-584dbcc88071 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.174135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.174763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.175048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.175361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.176017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.177616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57646a63-3082-470f-b60f-9c33d88a2886 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.188824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1235/2455] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:14.193308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.207714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7b791-bc32-4b1d-9ca2-9f8c52851a1b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.213089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.213783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.214068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.214348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] PATH_INFO: `/v3/projects/5e1395c40bbd42229b568a442627b886/users/91e30cef85bf4c13a7f6af4045d0ca91/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.230342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1395c40bbd42229b568a442627b886, user_id=91e30cef85bf4c13a7f6af4045d0ca91, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.233040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.246475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1221/2456] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:14 2022] PUT /identity/v3/projects/5e1395c40bbd42229b568a442627b886/users/91e30cef85bf4c13a7f6af4045d0ca91/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:14.258204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.258493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.258773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.284169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:14.287647 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] MFA Rules not processed for user `91e30cef85bf4c13a7f6af4045d0ca91`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:14.290259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:14.363574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1236/2457] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:14.375697 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.377468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:14.452374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6d0c-9f41-47c9-992a-4a8c5c75d389 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.458083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.458684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.458967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.459256 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.467824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.469399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ebf9-8adb-4658-9499-7932022fc2e5 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.478431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1222/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:14.499599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:14.501011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:14.583895 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380cbea-b8c9-475c-9637-d12e9764bf86 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:14.588264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:14.588867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:14.589149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:14.589426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:14.591792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:14.605794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:14.607501 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:14.618451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1237/2459] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:16.042922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd435969-966e-4e5d-8c3b-49626116156f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:16.043505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd435969-966e-4e5d-8c3b-49626116156f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:16.043505 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd435969-966e-4e5d-8c3b-49626116156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:16.062775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dd435969-966e-4e5d-8c3b-49626116156f None None] MFA Rules not processed for user `ea15538623ef4bacb42c4e03e2db72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:16.064030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd435969-966e-4e5d-8c3b-49626116156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:16.124276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1223/2460] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:17.323702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:17.391969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f2430-0291-4500-a99d-9a51942a6430 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:17.396971 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:17.397606 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:17.397889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:17.398198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] PATH_INFO: `/v3/users/91e30cef85bf4c13a7f6af4045d0ca91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:17.399370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e30cef85bf4c13a7f6af4045d0ca91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:17.401364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:17.438128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:17.456797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e30cef85bf4c13a7f6af4045d0ca91'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:17.464440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e30cef85bf4c13a7f6af4045d0ca91'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:17.465144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e30cef85bf4c13a7f6af4045d0ca91'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:17.472775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e30cef85bf4c13a7f6af4045d0ca91'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:17.485894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e30cef85bf4c13a7f6af4045d0ca91'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:17.491808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1238/2461] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:17 2022] DELETE /identity/v3/users/91e30cef85bf4c13a7f6af4045d0ca91 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:17.502100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084e56-0675-4106-b177-056fe198c1da None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:17.502655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084e56-0675-4106-b177-056fe198c1da None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:17.502655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084e56-0675-4106-b177-056fe198c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:17.521825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-00084e56-0675-4106-b177-056fe198c1da None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:17.523035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00084e56-0675-4106-b177-056fe198c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:17.553158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:17.584325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1224/2462] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:17.594179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:17.595630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:17.654265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201057-b2e4-4ee9-bc49-41463eb865a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:17.657999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:17.658598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:17.658882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:17.659160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:17.668967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:17.670498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043abcf-8636-4616-8b5c-ef711ed565ea service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:17.680508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1239/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:18.083849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00084e56-0675-4106-b177-056fe198c1da None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.086699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00084e56-0675-4106-b177-056fe198c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:18.099247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00084e56-0675-4106-b177-056fe198c1da None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.104264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.104982 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.105336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.105688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] PATH_INFO: `/v3/projects/5e1395c40bbd42229b568a442627b886` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.106958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1395c40bbd42229b568a442627b886)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.109008 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.165605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1395c40bbd42229b568a442627b886', 'request_id': 'req-3b7161af-f529-40d8-8bda-1a225de67540'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:18.171995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1395c40bbd42229b568a442627b886', 'request_id': 'req-3b7161af-f529-40d8-8bda-1a225de67540'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:18.182034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1395c40bbd42229b568a442627b886', 'request_id': 'req-3b7161af-f529-40d8-8bda-1a225de67540'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:18.196787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] The token cache is being invalidate because project 5e1395c40bbd42229b568a442627b886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:18.197160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:18.197835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b7161af-f529-40d8-8bda-1a225de67540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:18.199465 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1225/2464] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:18 2022] DELETE /identity/v3/projects/5e1395c40bbd42229b568a442627b886 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:18.204550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.204918 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.205268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.225686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:18.226962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:18.293922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1240/2465] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:18.301930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.302223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.302632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.323419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:18.324646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:18.332616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1226/2466] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:18.337929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.347727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a21ed8-6593-4006-9972-b062639049b3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.352407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.353015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.353308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.353577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.354258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:18.354562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.356219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.368476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1241/2467] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:18.372852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.383466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e38d87-1f85-4301-ab3f-bfbb243bc884 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.388852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.389461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.389747 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.390024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.390667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.392610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.423969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1227/2468] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:18 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:18.425440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.436515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047b60-0121-4249-b6e9-06ceef22df1b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.471074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:18.500480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.501099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.501380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.501679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.502520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.505181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.529148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1242/2469] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:18 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:18.533296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.544194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf041968-1082-4430-8994-ab97d9826a5e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.549202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.549807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.550095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.550374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.551027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.552674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.563846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1228/2470] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:27:18.568008 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.579758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550014-bcf3-4d75-8aef-4ef27255bd7c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.584676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.585286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.586061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.586388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.587133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.589290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1f3b1-fca2-4fc3-95d9-f8915a73c588 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.601019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1243/2471] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:27:18.605883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.616393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1724fe-d348-4c2f-93a5-ef060a9047a9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.621649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.622415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.622777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.623131 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] PATH_INFO: `/v3/projects/5cfc2ef99ba84ef6878109e703286434/users/2d4ef6884a904147b713ce0ae350a937/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.639600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cfc2ef99ba84ef6878109e703286434, user_id=2d4ef6884a904147b713ce0ae350a937, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.642336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.658130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1229/2472] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:18 2022] PUT /identity/v3/projects/5cfc2ef99ba84ef6878109e703286434/users/2d4ef6884a904147b713ce0ae350a937/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:18.669649 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.669952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.670306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.700911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] MFA Rules not processed for user `2d4ef6884a904147b713ce0ae350a937`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:18.702152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:18.775651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1244/2473] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:18.786720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:18.788560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:18.866354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad21c-c027-423f-af27-1d9c8d23c1bb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:18.872135 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:18.873047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:18.873423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:18.874099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:18.881269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:18.886358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:18.888419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae3df8-b504-4c4d-b83e-cb854e3130a6 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:18.900454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1230/2474] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:27.375146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:27.440924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fe2b2-5eff-4136-bd59-5b628cbbdd70 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:27.446196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:27.446814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:27.447094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:27.447398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] PATH_INFO: `/v3/users/2d4ef6884a904147b713ce0ae350a937` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:27.448566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4ef6884a904147b713ce0ae350a937)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:27.450488 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:27.497216 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:27.498969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4ef6884a904147b713ce0ae350a937'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:27.508616 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4ef6884a904147b713ce0ae350a937'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:27.509612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4ef6884a904147b713ce0ae350a937'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:27.521529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4ef6884a904147b713ce0ae350a937'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:27.536121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4ef6884a904147b713ce0ae350a937'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:27.544942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1245/2475] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:27 2022] DELETE /identity/v3/users/2d4ef6884a904147b713ce0ae350a937 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:27.549807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:27.550295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:27.550569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:27.582331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:27.583875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:27.654115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1231/2476] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:27.664287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:27.665794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:27.730194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19996-77a9-4dbd-b4a7-41758a17ae86 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:27.733875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:27.734487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:27.734770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:27.735189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:27.745806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:27.747365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8d34-9aba-462a-acf6-8fb1c5df7ea9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:27.759083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1246/2477] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:28.112204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.114347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:28.127740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc390205-2ff0-417a-a587-f0f17c948e1f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.133445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.134089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.134371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.134646 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] PATH_INFO: `/v3/projects/5cfc2ef99ba84ef6878109e703286434` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.135826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cfc2ef99ba84ef6878109e703286434)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.137778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.190939 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cfc2ef99ba84ef6878109e703286434', 'request_id': 'req-a2c5b053-ea9b-4f22-80ca-70a5638ea404'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:28.197627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cfc2ef99ba84ef6878109e703286434', 'request_id': 'req-a2c5b053-ea9b-4f22-80ca-70a5638ea404'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:28.203581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cfc2ef99ba84ef6878109e703286434', 'request_id': 'req-a2c5b053-ea9b-4f22-80ca-70a5638ea404'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:28.219673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] The token cache is being invalidate because project 5cfc2ef99ba84ef6878109e703286434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:28.219971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:28.220649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a2c5b053-ea9b-4f22-80ca-70a5638ea404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:28.222229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1232/2478] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:28 2022] DELETE /identity/v3/projects/5cfc2ef99ba84ef6878109e703286434 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:28.227852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.228132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.228411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.248192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:28.249497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:28.312175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1247/2479] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:27:28.320295 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.320585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.320867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.341987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:28.343293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:28.352101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1233/2480] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:28.356705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.366583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a161-48ea-4920-b564-e2aa7d582101 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.372220 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.372839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.373128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.373406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.374061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:28.374368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.376052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.388370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1248/2481] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:27:28.392855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.403193 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1076312-584a-4c73-99e1-68d50c079be7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.408312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.408914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.409203 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.409482 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.410137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.412136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.440608 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1234/2482] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:28.444994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.454997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b355c8-7d7a-47f6-abbc-ff43c9a18114 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.504923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:28.516751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.517367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.517656 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.517935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.518593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.520557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.549301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1249/2483] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:28.550637 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.563565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e4d0-d134-4bbb-a1b2-17e43d4df828 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.571204 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.572036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.572417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.572763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.573565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.575676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.591792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1235/2484] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:28.593052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.603525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42bd8-0a80-47e8-8a6d-06c6bab8655b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.608209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.608813 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.609103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.610043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.610043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.611931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c8289-e88c-4ea9-b185-bdaad7ae1ffe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.625515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1250/2485] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:28.627407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.638231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b601b4-3fdc-4e0d-adf9-2ec356005d1e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.644169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.645139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.645139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.645411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] PATH_INFO: `/v3/projects/2fb67192258a40d7bfc3a32246990fbf/users/0f447ce0b4de4e83bbf443009b9174f0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.663003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb67192258a40d7bfc3a32246990fbf, user_id=0f447ce0b4de4e83bbf443009b9174f0, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.666344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.681981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1236/2486] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:28 2022] PUT /identity/v3/projects/2fb67192258a40d7bfc3a32246990fbf/users/0f447ce0b4de4e83bbf443009b9174f0/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:28.693280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.693785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.694073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.728486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] MFA Rules not processed for user `0f447ce0b4de4e83bbf443009b9174f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:28.729667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:28.800772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1251/2487] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:28.806058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.825389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:28.877744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c3ae-5dd2-4e93-ba4e-4b7f287f7722 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:28.886882 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:28.887600 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:28.887891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:28.888176 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:28.888841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:28.890787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:28.925962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1237/2488] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:28.930408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:28.942100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fbca5-233f-4db2-bca0-99acb8187b85 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.012329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.012995 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.013283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.013602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.014272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.016486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.047510 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1252/2489] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:28 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:27:29.052800 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.065220 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672841c-f03f-4bb3-9a0c-e3c5c3fe4819 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.070404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.071085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.071410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.071834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.072508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.074496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.094053 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1238/2490] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:27:29.095040 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.109326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479845e-5f71-4f2d-8afd-9f27bf2d0643 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.115530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.116471 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.116611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.116787 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] PATH_INFO: `/v3/projects/649b624bee8746bca2922bde09021b8a/users/8f94ee702bc540958f73e694197b478a/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.136546 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649b624bee8746bca2922bde09021b8a, user_id=8f94ee702bc540958f73e694197b478a, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.139214 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.154700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1253/2491] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:29 2022] PUT /identity/v3/projects/649b624bee8746bca2922bde09021b8a/users/8f94ee702bc540958f73e694197b478a/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:29.158847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.170052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d746-4a75-4568-a01a-cbc0d3554859 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.238529 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.239545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.240042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.240517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.241766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.243789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b28c2-071c-4be5-9b2f-7849888e86b9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.261665 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1239/2492] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:27:29.266297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.283282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7ab-2c57-45db-80ed-9f2139982dc6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.291939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.292932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.293429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.293920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] PATH_INFO: `/v3/projects/649b624bee8746bca2922bde09021b8a/users/8f94ee702bc540958f73e694197b478a/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.298385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=649b624bee8746bca2922bde09021b8a, user_id=8f94ee702bc540958f73e694197b478a, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.304902 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.325928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1254/2493] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:29 2022] PUT /identity/v3/projects/649b624bee8746bca2922bde09021b8a/users/8f94ee702bc540958f73e694197b478a/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:29.337980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.338288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.338551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.369434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] MFA Rules not processed for user `8f94ee702bc540958f73e694197b478a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:29.370799 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:29.451680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1240/2494] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:29.459911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.464668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:29.485286 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:29.541489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099e256-3f5d-4dad-8ade-995259c9e87a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.548152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.548809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.549094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.549371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.562896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.564367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2c790-2e69-45b3-87e6-f1f4ca58a399 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.579331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1255/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:29.680099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:29.696744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:29.764910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690023c-7483-4e1e-a8b9-1ae145abe93f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:29.769541 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:29.770177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.770460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.770734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] PATH_INFO: `/v3/users/0f447ce0b4de4e83bbf443009b9174f0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.771878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f447ce0b4de4e83bbf443009b9174f0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:29.773861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:29.825274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f447ce0b4de4e83bbf443009b9174f0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:29.832238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f447ce0b4de4e83bbf443009b9174f0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:29.832906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f447ce0b4de4e83bbf443009b9174f0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:29.833517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f447ce0b4de4e83bbf443009b9174f0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:29.841499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f447ce0b4de4e83bbf443009b9174f0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:29.861474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1241/2496] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:29 2022] DELETE /identity/v3/users/0f447ce0b4de4e83bbf443009b9174f0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:29.866043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:29.866335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:29.867113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:29.890597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:29.892003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:29.982212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1256/2497] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:29.999650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:30.001418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:30.089713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb886b4-d472-434a-95a3-8b59ba0ae03b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:30.093411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:30.094328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:30.094610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:30.095456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:30.106773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:30.108579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:30.122298 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1242/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:30.400654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:30.402423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:30.416541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e6fea-8815-4b71-a408-96c8ef212fa3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:30.421235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:30.422432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:30.422897 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:30.423182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] PATH_INFO: `/v3/projects/2fb67192258a40d7bfc3a32246990fbf` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:30.424529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb67192258a40d7bfc3a32246990fbf)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:30.426447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:30.473875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb67192258a40d7bfc3a32246990fbf', 'request_id': 'req-1b068b6f-f410-4830-befe-503838ca88dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.481949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb67192258a40d7bfc3a32246990fbf', 'request_id': 'req-1b068b6f-f410-4830-befe-503838ca88dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.488679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb67192258a40d7bfc3a32246990fbf', 'request_id': 'req-1b068b6f-f410-4830-befe-503838ca88dc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.499436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] The token cache is being invalidate because project 2fb67192258a40d7bfc3a32246990fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:30.499727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.500367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.501935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1257/2499] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:30 2022] DELETE /identity/v3/projects/2fb67192258a40d7bfc3a32246990fbf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:30.506421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:30.516266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccab3e-41bd-4d5d-be7f-98f446e05fda service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:30.546978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:30.574474 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:30.575015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:30.575327 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:30.575605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] PATH_INFO: `/v3/users/8f94ee702bc540958f73e694197b478a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:30.576824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f94ee702bc540958f73e694197b478a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:30.578651 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:30.625138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f94ee702bc540958f73e694197b478a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.632634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f94ee702bc540958f73e694197b478a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.633322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f94ee702bc540958f73e694197b478a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.633926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f94ee702bc540958f73e694197b478a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.641560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c5ae827-8c3f-419e-bc8d-c555acf61417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f94ee702bc540958f73e694197b478a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:30.659706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1243/2500] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:30 2022] DELETE /identity/v3/users/8f94ee702bc540958f73e694197b478a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:30.926965 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:30.928514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:30.944029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b068b6f-f410-4830-befe-503838ca88dc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:30.987398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:31.012271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.013098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.013389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.013672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] PATH_INFO: `/v3/projects/649b624bee8746bca2922bde09021b8a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.014836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=649b624bee8746bca2922bde09021b8a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.016823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.064617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649b624bee8746bca2922bde09021b8a', 'request_id': 'req-d1d16da5-8d65-460e-92fd-2f26894264ae'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:31.071808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649b624bee8746bca2922bde09021b8a', 'request_id': 'req-d1d16da5-8d65-460e-92fd-2f26894264ae'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:31.078097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649b624bee8746bca2922bde09021b8a', 'request_id': 'req-d1d16da5-8d65-460e-92fd-2f26894264ae'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:31.088450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] The token cache is being invalidate because project 649b624bee8746bca2922bde09021b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:31.088744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:31.089386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1d16da5-8d65-460e-92fd-2f26894264ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:31.090866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1258/2501] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:30 2022] DELETE /identity/v3/projects/649b624bee8746bca2922bde09021b8a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:31.095669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.096178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.096280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.117919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:31.119058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:31.180439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1244/2502] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:31.188453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.188855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.189016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.209917 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:31.211419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:31.219314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1259/2503] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:27:31.223776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.234322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9799-28d5-4411-8ead-e98d3c2a0a70 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.239056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.239686 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.239972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.240255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.240895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:31.241198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.242807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.254796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1245/2504] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:31.259284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.268474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a43f3a-00d6-437a-9386-367f2dc6f8fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.273638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.274403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.274667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.274939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.275635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.277631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.311081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1260/2505] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:31.312923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.322956 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36014736-8e92-4391-821b-cc6f5e4a7e0c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.361396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:31.381385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.381944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.382247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.382522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.383145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.385098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.406324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1246/2506] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:31.410555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.421367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c566579-ae0c-409b-84cc-c159fcdbbd0f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.425920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.426505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.426781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.427051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.427786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.429365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.440283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1261/2507] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:31.444353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.454283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b303-6a8b-4c27-8503-9752ff85e5c9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.458874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.459545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.459829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.460618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.461264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.462871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5359bbd-92e7-4f37-8e78-a0435b827e2d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.473185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1247/2508] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:31.478044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.488058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f46dd-1a43-46dc-a63f-2fcccaa444c4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.492686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.493351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.493631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.493904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] PATH_INFO: `/v3/projects/59c992f8bc5b4ee6b6c00c6c146271f0/users/7cf3786c87184afab3155f0883c1ee8c/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.508823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c992f8bc5b4ee6b6c00c6c146271f0, user_id=7cf3786c87184afab3155f0883c1ee8c, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.511588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.525537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1262/2509] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:31 2022] PUT /identity/v3/projects/59c992f8bc5b4ee6b6c00c6c146271f0/users/7cf3786c87184afab3155f0883c1ee8c/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:31.535823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.536116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.536397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.564589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] MFA Rules not processed for user `7cf3786c87184afab3155f0883c1ee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:31.565872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:31.630942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1248/2510] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:27:31.636097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.697866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8e6a3-3f1b-495e-8a54-382b93a1cdeb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.702498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.703107 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.703430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.703705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.704331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.706264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.717679 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:31.735241 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1263/2511] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:31.739468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.750262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598e29b-62ec-4733-8d1f-03289a199c71 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.806447 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.807062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.807378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.807658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.808311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.810249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.832040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1249/2512] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:31 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:31.836663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.846959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34152-14b9-41ff-9bd5-ff8b116480b7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.851602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.852199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.852487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.852764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.853411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.855021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.864880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1264/2513] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:31.868994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.879028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463ba95-c2f4-473d-ae48-fc766e54a092 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.884852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.885345 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.885631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.885910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] PATH_INFO: `/v3/projects/22b6c02502ed4f1da1433453c902f32d/users/321e29b706a54272a425b839de02a364/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.901704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b6c02502ed4f1da1433453c902f32d, user_id=321e29b706a54272a425b839de02a364, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.904649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:31.917740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1250/2514] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:31 2022] PUT /identity/v3/projects/22b6c02502ed4f1da1433453c902f32d/users/321e29b706a54272a425b839de02a364/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:31.921752 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:31.932321 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374034e0-54bc-438c-a4b2-beccc3a4c5d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:31.987320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:31.987924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:31.988207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:31.988482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:31.989133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:31.990746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ca28b-3d3c-4e78-9f20-bfecf7d566fa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.001926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1265/2515] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:32.007779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.018713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9604ba-ffa1-461a-98ea-62fd2938689d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.023402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.024061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.024406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.024618 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] PATH_INFO: `/v3/projects/22b6c02502ed4f1da1433453c902f32d/users/321e29b706a54272a425b839de02a364/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.026601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b6c02502ed4f1da1433453c902f32d, user_id=321e29b706a54272a425b839de02a364, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.029265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.035758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:32.041654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1251/2516] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:32 2022] PUT /identity/v3/projects/22b6c02502ed4f1da1433453c902f32d/users/321e29b706a54272a425b839de02a364/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:32.056146 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.056438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.056722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.085255 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] MFA Rules not processed for user `321e29b706a54272a425b839de02a364`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:32.086489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:32.154880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1266/2517] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:32.167013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.168772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:32.227436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff10d3c-eb92-4390-86c5-a5cfcb2692ca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.231069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.231701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.231983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.232254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.241197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.243160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.252584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1252/2518] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:32.558914 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.568355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fa2fd-1c74-46bf-8249-2bdc65024b17 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.573464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '321e29b706a54272a425b839de02a364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22b6c02502ed4f1da1433453c902f32d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.574069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.574371 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.574636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.575346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:32.575660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.577299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.590319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1267/2519] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:32.594783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.604320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872cf23-b98d-44d3-b0ad-35a8a5adcf06 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.608997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '321e29b706a54272a425b839de02a364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22b6c02502ed4f1da1433453c902f32d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.609597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.609885 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.610161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.610792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.612726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.640028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1253/2520] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:32 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:32.644308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.655000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c292-a270-4c9d-9aa7-9b5ace0b6cb4 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.699122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:32.711739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '321e29b706a54272a425b839de02a364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22b6c02502ed4f1da1433453c902f32d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.712344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.712632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.712912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.713562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:32.713875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.715508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.727238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1268/2521] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:32.764638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:32.775087 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48e4ec-750e-4a22-8413-fee64d5ea405 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:32.779577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '321e29b706a54272a425b839de02a364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22b6c02502ed4f1da1433453c902f32d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:32.780200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:32.780491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:32.780772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] PATH_INFO: `/v3/projects/f148a3adf28444f9b80fcc502a564129` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:32.781910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f148a3adf28444f9b80fcc502a564129)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:32.783866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:32.830367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f148a3adf28444f9b80fcc502a564129', 'request_id': 'req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:32.837051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f148a3adf28444f9b80fcc502a564129', 'request_id': 'req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:32.842521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f148a3adf28444f9b80fcc502a564129', 'request_id': 'req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:32.854719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] The token cache is being invalidate because project f148a3adf28444f9b80fcc502a564129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:32.854997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:32.855645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c79108ef-7e6a-482d-9d59-0fe613e5ee5a tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:32.857148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1254/2522] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:32 2022] DELETE /identity/v3/projects/f148a3adf28444f9b80fcc502a564129 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:33.046879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:33.048619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:33.118398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fad4a-1b54-4a7f-9299-b6c3fee2fc31 tempest-VolumeQuotaClassesTest-1710771172 tempest-VolumeQuotaClassesTest-1710771172-project-admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:33.174552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:33.175176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:33.175489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:33.175764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] PATH_INFO: `/v3/users/7cf3786c87184afab3155f0883c1ee8c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:33.176884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf3786c87184afab3155f0883c1ee8c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:33.179445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:33.225132 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf3786c87184afab3155f0883c1ee8c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.232607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf3786c87184afab3155f0883c1ee8c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.233273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf3786c87184afab3155f0883c1ee8c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.240529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf3786c87184afab3155f0883c1ee8c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.253194 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf3786c87184afab3155f0883c1ee8c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.259343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1269/2523] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:33 2022] DELETE /identity/v3/users/7cf3786c87184afab3155f0883c1ee8c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:33.263923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:33.264413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:33.264544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:33.283140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:33.284378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:33.343290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1255/2524] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:33.354007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:33.355432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:33.367135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:33.416957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d300a98-7e52-462b-8f55-0ee228f21b6c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:33.420638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:33.421238 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:33.421519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:33.421791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:33.431181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:33.432699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:33.441931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1270/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:33.717103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:33.718639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:33.727629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:33.732178 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364a34a-4eb4-4fa9-beb1-baf5c5612d3a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:33.738073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:33.738446 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:33.738924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:33.739772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] PATH_INFO: `/v3/projects/59c992f8bc5b4ee6b6c00c6c146271f0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:33.740890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c992f8bc5b4ee6b6c00c6c146271f0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:33.743400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:33.792764 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c992f8bc5b4ee6b6c00c6c146271f0', 'request_id': 'req-df6d0c87-3b2a-458e-9bb5-e75602325bca'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.799083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c992f8bc5b4ee6b6c00c6c146271f0', 'request_id': 'req-df6d0c87-3b2a-458e-9bb5-e75602325bca'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.804828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c992f8bc5b4ee6b6c00c6c146271f0', 'request_id': 'req-df6d0c87-3b2a-458e-9bb5-e75602325bca'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.817325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] The token cache is being invalidate because project 59c992f8bc5b4ee6b6c00c6c146271f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:33.817644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.818288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.820319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1256/2526] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:33 2022] DELETE /identity/v3/projects/59c992f8bc5b4ee6b6c00c6c146271f0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:33.824324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:33.833881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09540e71-5ee4-4449-b816-5adf65f7372d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:33.889603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:33.890228 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:33.890508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:33.890781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] PATH_INFO: `/v3/users/321e29b706a54272a425b839de02a364` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:33.891923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] RBAC: Authorizing `identity:delete_user(user_id=321e29b706a54272a425b839de02a364)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:33.893863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:33.937727 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321e29b706a54272a425b839de02a364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.944980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321e29b706a54272a425b839de02a364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.945652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321e29b706a54272a425b839de02a364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.952201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321e29b706a54272a425b839de02a364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.964751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7947510d-21f8-4333-8c2a-6bd4e0ffe1be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321e29b706a54272a425b839de02a364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:33.970534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1271/2527] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:33 2022] DELETE /identity/v3/users/321e29b706a54272a425b839de02a364 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:34.236692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.238114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:34.251975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d0c87-3b2a-458e-9bb5-e75602325bca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.310575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.311307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.311516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.311794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] PATH_INFO: `/v3/projects/22b6c02502ed4f1da1433453c902f32d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.313035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b6c02502ed4f1da1433453c902f32d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.314884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.361335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b6c02502ed4f1da1433453c902f32d', 'request_id': 'req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:34.368062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b6c02502ed4f1da1433453c902f32d', 'request_id': 'req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:34.373631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b6c02502ed4f1da1433453c902f32d', 'request_id': 'req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:34.388281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] The token cache is being invalidate because project 22b6c02502ed4f1da1433453c902f32d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:34.388576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:34.389231 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84b3b345-bfb6-4f2b-b533-cf6e4997b9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:34.390730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1257/2528] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:34 2022] DELETE /identity/v3/projects/22b6c02502ed4f1da1433453c902f32d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:34.395869 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.396164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.396444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.416623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:34.417841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:34.443749 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:34.478537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1272/2529] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:34.486262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.486553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.486860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.508084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:34.509291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:34.517423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1258/2530] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:34.521644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.531532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72351d6-9a7d-49f0-be30-15c5643a05c2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.536280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.536880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.537157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.537426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.538069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:34.538370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.540058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.551694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1273/2531] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:34.556647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.565989 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e10c-7b1e-4ca9-ad73-af9d2d18aa18 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.571431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.572023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.572304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.572578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.573206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.575088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.585166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:34.606059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1259/2532] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:34.610566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.627703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af75b-0a5b-4650-8ccd-d3599c3e19d3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.695116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.695745 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.696027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.696298 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.696934 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.698833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.720096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1274/2533] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:27:34.724112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.734738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb6795-a32f-4088-b577-f97c3430a3a0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.739382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.739969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.740248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.740519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.741148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.743409 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.753166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1260/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:34.757479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.767526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab81a-3035-4d4b-a47a-b936eed269cc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.772237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.772837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.773121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.773397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.774047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.775691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d8326-336e-449d-ab28-6234da7f6edd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.786304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1275/2535] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:34.790870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:34.800840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0967f28-6a3c-4886-8ca7-d83e047f889b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:34.806197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:34.806823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.807129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.807484 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] PATH_INFO: `/v3/projects/2a258303ea1d468e9af1541254cfc7de/users/6efdf27f15e6431fbe14693d7f29150f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.822161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a258303ea1d468e9af1541254cfc7de, user_id=6efdf27f15e6431fbe14693d7f29150f, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:34.824679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:34.837088 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1261/2536] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:34 2022] PUT /identity/v3/projects/2a258303ea1d468e9af1541254cfc7de/users/6efdf27f15e6431fbe14693d7f29150f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:34.848313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:34.848812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:34.848943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:34.877252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] MFA Rules not processed for user `6efdf27f15e6431fbe14693d7f29150f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:34.878478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:34.947759 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1276/2537] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:34.952255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.022384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac53b1f-2d64-48b3-8775-aa25bc51ceba admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.028010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.028625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.028913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.029195 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.029831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.031894 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.060183 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1262/2538] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:34 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:35.064159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.074955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f167-57d5-4106-a29d-e6df9f544eb4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.099866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:35.133915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.134509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.134793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.135069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.135757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.137688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.160431 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1277/2539] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:35 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:35.168785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.179502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd445-bbfc-4b9d-bedb-e007b295775b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.184229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.184754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.185031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.185304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.185943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.187569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.198029 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1263/2540] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:35.202162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.213013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e536ca-4190-4e32-a5d0-4eeb5f1ff532 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.217845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.218553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.218836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.219115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] PATH_INFO: `/v3/projects/5077b1c8cc12413db757af535b00d796/users/cd669aa580094d05aa9f10521ed6cc30/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.234998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5077b1c8cc12413db757af535b00d796, user_id=cd669aa580094d05aa9f10521ed6cc30, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.237790 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.251644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1278/2541] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:35 2022] PUT /identity/v3/projects/5077b1c8cc12413db757af535b00d796/users/cd669aa580094d05aa9f10521ed6cc30/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:35.255953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.265829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399724-5624-4c82-b987-fcf3f49c851d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.332072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.332744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.332847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.333107 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.333742 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.336017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97cfc4-cda7-42a7-adf9-1c6ade493aee admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.345841 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1264/2542] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:35.350473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.361444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa66fd-f4be-4203-999c-ef9e8edf608f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.366173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.366828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.367111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.367424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] PATH_INFO: `/v3/projects/5077b1c8cc12413db757af535b00d796/users/cd669aa580094d05aa9f10521ed6cc30/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.369362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5077b1c8cc12413db757af535b00d796, user_id=cd669aa580094d05aa9f10521ed6cc30, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.372068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.383276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1279/2543] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:35 2022] PUT /identity/v3/projects/5077b1c8cc12413db757af535b00d796/users/cd669aa580094d05aa9f10521ed6cc30/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:35.393267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.393555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.393833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.422883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] MFA Rules not processed for user `cd669aa580094d05aa9f10521ed6cc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:35.424246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:35.436903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:35.494028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1265/2544] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:35.505042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:35.507424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:35.566713 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd0dd-6357-4634-9c3e-b7eb5cf1f46b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:35.570900 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:35.571541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:35.571827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:35.572102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:35.581179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:35.582686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b6af4-7b5b-4a17-b11b-5d2db7454d35 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:35.593058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1280/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:35.967100 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:36.029004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d2355-557b-4074-b830-ee47ce721e6f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:36.033586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:36.034232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:36.034497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:36.034774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] PATH_INFO: `/v3/users/6efdf27f15e6431fbe14693d7f29150f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:36.035910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efdf27f15e6431fbe14693d7f29150f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:36.038125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:36.084510 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efdf27f15e6431fbe14693d7f29150f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.091562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efdf27f15e6431fbe14693d7f29150f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.092165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efdf27f15e6431fbe14693d7f29150f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.092775 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efdf27f15e6431fbe14693d7f29150f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.099855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efdf27f15e6431fbe14693d7f29150f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.119459 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1266/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:35 2022] DELETE /identity/v3/users/6efdf27f15e6431fbe14693d7f29150f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:36.124050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:36.124557 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:36.124659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:36.143536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:36.144740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:36.205034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1281/2547] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:36.215477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:36.217133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:36.250189 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:36.278467 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5897d49-d8c5-4e75-907b-cc41ac5e95dc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:36.282075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:36.282673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:36.282953 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:36.283269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:36.292701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:36.294104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:36.303394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1267/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:36.575056 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:36.576549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:36.589118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758086-baa7-4c87-b14b-84ac6af12e04 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:36.593905 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:36.594523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:36.594804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:36.595080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] PATH_INFO: `/v3/projects/2a258303ea1d468e9af1541254cfc7de` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:36.596247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a258303ea1d468e9af1541254cfc7de)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:36.598489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:36.616912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:36.647949 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a258303ea1d468e9af1541254cfc7de', 'request_id': 'req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.655765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a258303ea1d468e9af1541254cfc7de', 'request_id': 'req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.662459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a258303ea1d468e9af1541254cfc7de', 'request_id': 'req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.673097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] The token cache is being invalidate because project 2a258303ea1d468e9af1541254cfc7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:36.673390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.674033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.675561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1282/2549] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:36 2022] DELETE /identity/v3/projects/2a258303ea1d468e9af1541254cfc7de => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:36.680676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:36.697619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f5c25-95cd-4f0a-aece-b5eafc9391eb service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:36.754840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:36.755492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:36.755778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:36.756054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] PATH_INFO: `/v3/users/cd669aa580094d05aa9f10521ed6cc30` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:36.757774 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd669aa580094d05aa9f10521ed6cc30)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:36.759726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:36.805292 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd669aa580094d05aa9f10521ed6cc30'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.812323 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd669aa580094d05aa9f10521ed6cc30'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.812985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd669aa580094d05aa9f10521ed6cc30'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.813588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd669aa580094d05aa9f10521ed6cc30'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.820880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0545ee25-8b5f-424f-b30e-273c5def97fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd669aa580094d05aa9f10521ed6cc30'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:36.838160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1268/2550] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:36 2022] DELETE /identity/v3/users/cd669aa580094d05aa9f10521ed6cc30 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:37.102848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.104349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:37.118648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08617aa2-8f2e-49bf-a9bc-fbf53ded12c1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.176418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.177070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.177323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.177601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] PATH_INFO: `/v3/projects/5077b1c8cc12413db757af535b00d796` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.179345 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5077b1c8cc12413db757af535b00d796)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.181284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.230346 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5077b1c8cc12413db757af535b00d796', 'request_id': 'req-8af6914a-1c18-41df-8bad-a39909e4f57a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:37.237704 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5077b1c8cc12413db757af535b00d796', 'request_id': 'req-8af6914a-1c18-41df-8bad-a39909e4f57a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:37.244033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5077b1c8cc12413db757af535b00d796', 'request_id': 'req-8af6914a-1c18-41df-8bad-a39909e4f57a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:37.254409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] The token cache is being invalidate because project 5077b1c8cc12413db757af535b00d796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:37.254946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:37.255609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8af6914a-1c18-41df-8bad-a39909e4f57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:37.257109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1283/2551] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:37 2022] DELETE /identity/v3/projects/5077b1c8cc12413db757af535b00d796 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:37.263019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.263340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.263628 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.283886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:37.285099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:37.329692 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:37.346023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1269/2552] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:37.354689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.354981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.355294 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.374875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:37.376076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:37.384423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1284/2553] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:37.388488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.398832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d331446-79fe-4c54-a914-32732b5176bb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.403504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.404106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.404391 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.404668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.405311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:37.405619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.407245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.419498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1270/2554] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:37.423782 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.433033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc8dda-daa2-4ecd-a41f-3ca8da0f277d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.437734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.438344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.438609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.438887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.439551 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.441466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.468536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1285/2555] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:37 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:37.472932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.483624 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684516d-39cc-413b-865f-eff5ad678df8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.541560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.542166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.542451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.542730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.543404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.545341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.565914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1271/2556] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:37 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:37.570720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.581072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f3e85-f894-4365-80d0-cff2bdb5e094 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.586349 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.586945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.587262 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.587538 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.588182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.589793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.591154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:37.601503 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1286/2557] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:37.605347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.615740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518135c-820c-49d5-a299-fa2d696c23b0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.620428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.621057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.621347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.621622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.622264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.624064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47ca58-d586-4b2f-9946-5a450fa71b8e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.635849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1272/2558] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:37.639924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.651025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e99d-579e-4ceb-b833-86fee6f972f9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.655668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.656316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.656594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.656887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] PATH_INFO: `/v3/projects/115dabcc96e94d19aa07a9f75204f2e0/users/3e057df513454d7d92f05c23319a461d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.671293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115dabcc96e94d19aa07a9f75204f2e0, user_id=3e057df513454d7d92f05c23319a461d, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.673908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.686864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1287/2559] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:37 2022] PUT /identity/v3/projects/115dabcc96e94d19aa07a9f75204f2e0/users/3e057df513454d7d92f05c23319a461d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:37.698478 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.698984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.699091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.728301 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] MFA Rules not processed for user `3e057df513454d7d92f05c23319a461d`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:37.729530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:37.794379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1273/2560] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:37.806289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:37.808088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:37.864379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30595574-1030-41e2-bc6a-928bad43cb79 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:37.868117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:37.868760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:37.869047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:37.869325 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:37.877728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:37.879181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:37.889600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1288/2561] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:43.499319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:43.500758 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:43.513020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a027dc-b3b8-4751-b6db-817d7486a3a9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:43.516854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:43.517442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:43.517728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:43.518001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:43.519830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:43.566361 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:43.601166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:43.602609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76840d4-72a5-4772-b4a1-8cfb1cc80acf service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:43.620740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1274/2562] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:47.012233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:47.085077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6a491-038e-4934-b696-26551ad4320c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:47.090417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:47.091062 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:47.091377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:47.091655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] PATH_INFO: `/v3/users/3e057df513454d7d92f05c23319a461d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:47.092824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e057df513454d7d92f05c23319a461d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:47.094812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:47.144520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e057df513454d7d92f05c23319a461d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.152453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e057df513454d7d92f05c23319a461d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.153159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e057df513454d7d92f05c23319a461d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.161005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e057df513454d7d92f05c23319a461d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.171530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:47.174037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e057df513454d7d92f05c23319a461d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.181849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1289/2563] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:47 2022] DELETE /identity/v3/users/3e057df513454d7d92f05c23319a461d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:47.186437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:47.186919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:47.187017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:47.206490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:47.207872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:47.273638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1275/2564] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:47.286688 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:47.289090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:47.379378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf210-1dba-4861-bf00-ecd8b6c2848f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:47.384085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:47.384791 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:47.385061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:47.385330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:47.396391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:47.398462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:47.410041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1290/2565] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:27:47.738691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:47.740215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:47.756629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9d7a3-e044-4a25-8803-b78c9a71377e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:47.763166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:47.765940 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:47.768714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:47.769014 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] PATH_INFO: `/v3/projects/115dabcc96e94d19aa07a9f75204f2e0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:47.770307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115dabcc96e94d19aa07a9f75204f2e0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:47.773722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:47.862171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115dabcc96e94d19aa07a9f75204f2e0', 'request_id': 'req-dd124e1b-2465-4459-8e8f-1f7ba29d3560'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.872553 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115dabcc96e94d19aa07a9f75204f2e0', 'request_id': 'req-dd124e1b-2465-4459-8e8f-1f7ba29d3560'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.880828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115dabcc96e94d19aa07a9f75204f2e0', 'request_id': 'req-dd124e1b-2465-4459-8e8f-1f7ba29d3560'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.896338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] The token cache is being invalidate because project 115dabcc96e94d19aa07a9f75204f2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:47.896615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.897402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:47.899958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1276/2566] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:47 2022] DELETE /identity/v3/projects/115dabcc96e94d19aa07a9f75204f2e0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:50.669684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:50.671080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:50.682276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855359-6d1c-47d5-a88a-350967198697 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:50.729916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:50.730511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:50.730792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:50.731066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:50.732801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:50.795054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:50.796587 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:50.808403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1291/2567] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:54.024621 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.026789 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:54.040263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124e1b-2465-4459-8e8f-1f7ba29d3560 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.045601 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.046230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.046515 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.046794 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] PATH_INFO: `/v3/users/c3dc3a92c6424cdbb3a2d3c1baaa95e3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.047945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3dc3a92c6424cdbb3a2d3c1baaa95e3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.049898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.087238 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:54.098552 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3dc3a92c6424cdbb3a2d3c1baaa95e3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.106328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3dc3a92c6424cdbb3a2d3c1baaa95e3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.107032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3dc3a92c6424cdbb3a2d3c1baaa95e3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.107672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3dc3a92c6424cdbb3a2d3c1baaa95e3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.114863 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8e5f2cae-856e-4b09-9ad9-0598c20389d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3dc3a92c6424cdbb3a2d3c1baaa95e3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.136148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1277/2568] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:27:54 2022] DELETE /identity/v3/users/c3dc3a92c6424cdbb3a2d3c1baaa95e3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:54.311603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.322399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7488-1c1f-488d-a86c-02bd0be0e03e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.369824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:54.383337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.383975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.384263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.385199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] PATH_INFO: `/v3/projects/d07b587d6ac148b0a7751c851260b3bc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.386654 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07b587d6ac148b0a7751c851260b3bc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.388437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.434756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07b587d6ac148b0a7751c851260b3bc', 'request_id': 'req-142a674a-789f-4955-a9fe-39f2bc5d82a9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.443159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07b587d6ac148b0a7751c851260b3bc', 'request_id': 'req-142a674a-789f-4955-a9fe-39f2bc5d82a9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.450096 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07b587d6ac148b0a7751c851260b3bc', 'request_id': 'req-142a674a-789f-4955-a9fe-39f2bc5d82a9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.460435 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] The token cache is being invalidate because project d07b587d6ac148b0a7751c851260b3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:27:54.460725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.461351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-142a674a-789f-4955-a9fe-39f2bc5d82a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:27:54.463473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1292/2569] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:27:54 2022] DELETE /identity/v3/projects/d07b587d6ac148b0a7751c851260b3bc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:27:54.468516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.468809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.469118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.488713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:54.489931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:54.549169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1278/2570] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:54.557258 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.557548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.557827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.578143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:54.579426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:54.587526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1293/2571] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:54.592537 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.602281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef5e4-9c0e-4124-bca8-79bf58d3720f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.606948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.607551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.607838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.608115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.608765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:27:54.609073 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.610660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.622296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1279/2572] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:27:54.626804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.636929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c6655-e6b5-4af9-bcb5-58eb8b3d8f65 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.641541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.642140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.642421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.642695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.643357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.645279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.672799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1294/2573] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:54 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:54.677158 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.687169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec8997-9429-426b-b9f2-3d550b400d7a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.746967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.747617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.747904 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.748184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.748825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.750778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.772997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1280/2574] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:54 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:54.777409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.788046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1141-b4f7-47e2-a62d-bc20c3d6315c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.792981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.793593 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.793896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.794169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.794845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.796506 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.807738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1295/2575] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:54.811878 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.821803 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b8c6-1a1a-4e84-a9bd-d257e75f500e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.827181 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.827787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.828071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.828347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.828985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.830582 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7a61-1f8d-4d48-9c1b-70618f9a7fd9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.841357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1281/2576] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:54.845528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:54.856271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8873f-41cb-4372-8dfd-de906ae05055 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:54.860851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:54.861496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.861777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.862073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] PATH_INFO: `/v3/projects/9cec207759844bb483bc01e2da794419/users/8442b0b05646416290f00c9c4eb21943/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.878903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cec207759844bb483bc01e2da794419, user_id=8442b0b05646416290f00c9c4eb21943, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:54.881915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:54.896034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1296/2577] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:54 2022] PUT /identity/v3/projects/9cec207759844bb483bc01e2da794419/users/8442b0b05646416290f00c9c4eb21943/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:54.906609 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:54.907042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:54.907370 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:54.939357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] MFA Rules not processed for user `8442b0b05646416290f00c9c4eb21943`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:54.940567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:54.945585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:55.012784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1282/2578] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:55.017329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.080885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a479813-f17d-4fd8-bad8-5c673608c80e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.085596 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.086199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.086480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.086754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.087416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.089357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.108978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:27:55.117252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1297/2579] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:27:55 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:27:55.121762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.133034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056d6d7-2d4e-40ff-93e7-ceb36610415c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.189946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.190545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.190824 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.191096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.191756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.193661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.214331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1283/2580] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:27:55 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:27:55.218116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.229127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645448f-8e02-4550-ae18-e3a73ad5ab1c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.234174 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.234703 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.234989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.235317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.235973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.237586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.248888 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1298/2581] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:55.253095 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.263667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11773fbf-f232-4eae-8c91-ce7e695c4ade admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.268444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.269094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.269379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.269668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] PATH_INFO: `/v3/projects/9d08fcbc672249d68c99f102d4402408/users/f8c50e7129a548338bba0a479c16adfb/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.284439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d08fcbc672249d68c99f102d4402408, user_id=f8c50e7129a548338bba0a479c16adfb, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.287144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.300344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1284/2582] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:55 2022] PUT /identity/v3/projects/9d08fcbc672249d68c99f102d4402408/users/f8c50e7129a548338bba0a479c16adfb/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:55.304552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.315460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dac573-bf13-4d2c-acf5-d075bc9cda5f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.371956 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.372556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.372840 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.373118 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.373754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.375383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987092ce-2b0b-4ba3-9977-0c72a45b8479 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.385385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1299/2583] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:27:55.389622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.400127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89953c5-1de1-4abd-8b7d-0ee6b9a58394 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.405016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.405915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.406201 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.406477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] PATH_INFO: `/v3/projects/9d08fcbc672249d68c99f102d4402408/users/f8c50e7129a548338bba0a479c16adfb/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.408435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d08fcbc672249d68c99f102d4402408, user_id=f8c50e7129a548338bba0a479c16adfb, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.411068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.423247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1285/2584] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:27:55 2022] PUT /identity/v3/projects/9d08fcbc672249d68c99f102d4402408/users/f8c50e7129a548338bba0a479c16adfb/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:27:55.435756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.436045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.436326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.464425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] MFA Rules not processed for user `f8c50e7129a548338bba0a479c16adfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:27:55.465556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:55.535476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1300/2585] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:27:55.547396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.549128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:27:55.608412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858ecf-f959-4510-9d43-bfc21e46d26e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.612142 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.612751 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.613037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.613315 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.622354 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.623826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.633713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1286/2586] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:27:55.707271 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:27:55.717454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1240f72-bc46-493d-ae93-0b6f90a009b8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:27:55.721239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:27:55.721866 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:27:55.722151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:27:55.722432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:27:55.780358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:27:55.781814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:27:55.790700 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1301/2587] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:05.629156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:05.786619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:27.450941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:27.452842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:27.519116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936565b5-5f46-4823-a239-30f3a08cd9d8 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:27.522641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:27.523265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:27.523551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:27.523821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:27.526152 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:27.535397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:27.536805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0a493-6044-43c4-bd54-cdb20b260377 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:27.544696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1287/2588] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:37.541205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=26 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:39.344733 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:39.409629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e41e-6e83-4f30-81c8-92f1cf1a6e46 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:39.414248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:39.414868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:39.415152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:39.415460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] PATH_INFO: `/v3/users/8442b0b05646416290f00c9c4eb21943` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:39.416580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8442b0b05646416290f00c9c4eb21943)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:39.419011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:39.464872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8442b0b05646416290f00c9c4eb21943'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:39.471833 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8442b0b05646416290f00c9c4eb21943'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:39.472502 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8442b0b05646416290f00c9c4eb21943'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:39.479880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8442b0b05646416290f00c9c4eb21943'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:39.492296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8442b0b05646416290f00c9c4eb21943'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:39.501070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1302/2589] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:39 2022] DELETE /identity/v3/users/8442b0b05646416290f00c9c4eb21943 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:39.505793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:39.506398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:39.506677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:39.526167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:39.527393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:39.586132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1288/2590] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:39.596138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:39.597591 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:39.660907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf3e6-56e2-499d-8628-76063bc47b04 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:39.664693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:39.665310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:39.665595 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:39.666175 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:39.678581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:39.680072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:39.689762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1303/2591] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:39.960544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:39.962614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:39.976216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c2411-f05f-426a-b1f3-e5099d8e585e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:39.982855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:39.983607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:39.983821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:39.984132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] PATH_INFO: `/v3/projects/9cec207759844bb483bc01e2da794419` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:39.985232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cec207759844bb483bc01e2da794419)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:39.987149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.034429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cec207759844bb483bc01e2da794419', 'request_id': 'req-a735dea3-7a2c-4ea0-a224-e82183d7de69'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.040443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cec207759844bb483bc01e2da794419', 'request_id': 'req-a735dea3-7a2c-4ea0-a224-e82183d7de69'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.046889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cec207759844bb483bc01e2da794419', 'request_id': 'req-a735dea3-7a2c-4ea0-a224-e82183d7de69'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.059452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] The token cache is being invalidate because project 9cec207759844bb483bc01e2da794419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:40.059749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.060393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.061952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1289/2592] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:39 2022] DELETE /identity/v3/projects/9cec207759844bb483bc01e2da794419 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:40.065964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.076300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3ec8-d785-4180-b7be-8259263f479e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.095176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:40.135156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.135809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.136094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.136372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] PATH_INFO: `/v3/users/f8c50e7129a548338bba0a479c16adfb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.137988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8c50e7129a548338bba0a479c16adfb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.139952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.185911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8c50e7129a548338bba0a479c16adfb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.192883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8c50e7129a548338bba0a479c16adfb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.193613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8c50e7129a548338bba0a479c16adfb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.201137 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8c50e7129a548338bba0a479c16adfb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.213045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-02bc2cf9-99f7-40e8-a9d7-b684fd5f5594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8c50e7129a548338bba0a479c16adfb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.219563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1304/2593] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:40 2022] DELETE /identity/v3/users/f8c50e7129a548338bba0a479c16adfb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:40.490941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.492564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:40.508745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735dea3-7a2c-4ea0-a224-e82183d7de69 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.569835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.570419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.570703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.570981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] PATH_INFO: `/v3/projects/9d08fcbc672249d68c99f102d4402408` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.572136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d08fcbc672249d68c99f102d4402408)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.574118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.605027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:40.623840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d08fcbc672249d68c99f102d4402408', 'request_id': 'req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.630717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d08fcbc672249d68c99f102d4402408', 'request_id': 'req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.637779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d08fcbc672249d68c99f102d4402408', 'request_id': 'req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.650555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] The token cache is being invalidate because project 9d08fcbc672249d68c99f102d4402408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:40.650839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.651508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bbedc5ca-dfe9-4a75-8fbb-d487b28a0f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:40.653058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1290/2594] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:40 2022] DELETE /identity/v3/projects/9d08fcbc672249d68c99f102d4402408 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:40.657940 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.658432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.658564 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.678015 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:40.679237 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:40.737380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1305/2595] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:40.745262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.745554 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.745837 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.766750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:40.768005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:40.775945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1291/2596] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:40.781080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.791765 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b25c6-d60e-43da-b6fc-6830a51bc2cb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.797035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.797632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.797919 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.798197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.798852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:28:40.799159 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.800794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.812967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1306/2597] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:28:40.817314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.826557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fc531-beb5-46b9-8c8f-a1ecf45c719a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.831161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.831783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.832068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.832344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.832978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.834897 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.863488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1292/2598] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:40 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:28:40.867705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.878625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad5ce-5eec-4b7f-a985-ffaeae0aa10d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.905385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:40.936957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.937556 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.937837 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.938111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.938734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.940684 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.962050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1307/2599] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:40 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:40.965991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:40.976413 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327f683-78bc-46c8-a963-3c36066d5b60 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:40.981658 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:40.982255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:40.982540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:40.982820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:40.983516 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:40.985109 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:40.995058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1293/2600] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:40.999718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.010322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d365-f053-4e18-968c-902d7213fde5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.015016 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.015644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.015929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.016201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.016838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.018438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6597eea-bd9f-4312-a2f8-7e1a10e2ec40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.029122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1308/2601] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:41.033290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.043779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae1b8e-ce1b-43ff-bf4c-f27a3178fb0b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.048461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.049110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.049398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.049687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] PATH_INFO: `/v3/projects/6cff24799eb24abcb8f7506a2ebc972a/users/02dbdbce8e06412b879bb98cdaf9b85e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.064966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cff24799eb24abcb8f7506a2ebc972a, user_id=02dbdbce8e06412b879bb98cdaf9b85e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.067659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.080316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1294/2602] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:41 2022] PUT /identity/v3/projects/6cff24799eb24abcb8f7506a2ebc972a/users/02dbdbce8e06412b879bb98cdaf9b85e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:41.091844 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.092133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.092411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.120520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] MFA Rules not processed for user `02dbdbce8e06412b879bb98cdaf9b85e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:41.122324 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:41.187748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1309/2603] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:41.192320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.258464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271f44-63d4-47ac-953f-fcac73191178 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.263174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.263807 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.264094 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.264372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.265033 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.266996 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.294942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1295/2604] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:41 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:41.299115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.310024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3315c-6314-4e93-b1b2-c93ce0fa390e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.369461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.369987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.370273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.370549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.371243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.373189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.394280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1310/2605] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:41 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:41.398782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.409811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d403d-ca7e-426e-9838-36f0ce72c40b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.414531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.415161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.415615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.415809 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.416557 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.418767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.429464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1296/2606] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:41.433619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.443540 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29aa68-3a7a-49c0-bc68-ce97a05a0d78 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.448099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.448737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.449017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.449293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] PATH_INFO: `/v3/projects/172b4dca6d9b41a7b581be9336d2f18f/users/c3954dfd64de49d788a16f7d01043f64/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.464157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172b4dca6d9b41a7b581be9336d2f18f, user_id=c3954dfd64de49d788a16f7d01043f64, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.467382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.480306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1311/2607] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:41 2022] PUT /identity/v3/projects/172b4dca6d9b41a7b581be9336d2f18f/users/c3954dfd64de49d788a16f7d01043f64/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:41.484864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.494745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3d76-6dba-466e-a474-9734d11438b9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.514739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:41.553825 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.554436 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.554728 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.555007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.555673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.557266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e558ce-b537-4d36-81ce-d5cac0265026 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.566933 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1297/2608] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:41.571211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.577208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:41.584760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4586e03-2f8b-4332-bafd-a4018fa732a1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.589863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.591080 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.591459 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.591908 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] PATH_INFO: `/v3/projects/172b4dca6d9b41a7b581be9336d2f18f/users/c3954dfd64de49d788a16f7d01043f64/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.593928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172b4dca6d9b41a7b581be9336d2f18f, user_id=c3954dfd64de49d788a16f7d01043f64, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.596657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.606978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1312/2609] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:41 2022] PUT /identity/v3/projects/172b4dca6d9b41a7b581be9336d2f18f/users/c3954dfd64de49d788a16f7d01043f64/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:41.617153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.617450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.617730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.646594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] MFA Rules not processed for user `c3954dfd64de49d788a16f7d01043f64`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:41.647645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:41.716305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1298/2610] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:41.727525 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.729350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:41.787577 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16f07c-0c09-42b6-82df-86aac7cadd79 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.791300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.791930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.792215 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.792493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.802010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.803485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.812937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1313/2611] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:41.888967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:41.899047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21862ea5-a8d8-4ddd-a607-787c27b00db9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:41.903319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:41.903916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:41.904200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:41.904573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:41.962744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:41.965132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97952f-88c8-49a4-974d-2a16a48493d8 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:41.974338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1299/2612] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:44.713990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:44.776384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83965a3-37bb-4e22-8ecd-d563bf20c2ce service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:44.781603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:44.782233 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:44.782516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:44.782793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] PATH_INFO: `/v3/users/02dbdbce8e06412b879bb98cdaf9b85e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:44.783937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dbdbce8e06412b879bb98cdaf9b85e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:44.785868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:44.832758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dbdbce8e06412b879bb98cdaf9b85e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:44.840126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dbdbce8e06412b879bb98cdaf9b85e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:44.840795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dbdbce8e06412b879bb98cdaf9b85e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:44.849725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dbdbce8e06412b879bb98cdaf9b85e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:44.862295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dbdbce8e06412b879bb98cdaf9b85e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:44.868177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1314/2613] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:44 2022] DELETE /identity/v3/users/02dbdbce8e06412b879bb98cdaf9b85e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:44.872736 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:44.873240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:44.873342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:44.893197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:44.894415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:44.938634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:44.958110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1300/2614] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:28:44.968461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:44.969841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:44.983732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:45.043947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bcda3-5576-4e76-8d66-2031698fed05 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:45.048426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:45.049033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:45.049318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:45.049597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:45.058822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:45.060508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:45.069926 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1315/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:45.340594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:45.342114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:45.354027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37439345-53e6-4ed0-81e9-b4f301b1e79a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:45.358620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:45.359261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:45.359568 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:45.359840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] PATH_INFO: `/v3/projects/6cff24799eb24abcb8f7506a2ebc972a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:45.360949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cff24799eb24abcb8f7506a2ebc972a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:45.362867 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:45.410464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cff24799eb24abcb8f7506a2ebc972a', 'request_id': 'req-e84cf4a8-c298-4b59-8dfa-13c569bf8808'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.416726 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cff24799eb24abcb8f7506a2ebc972a', 'request_id': 'req-e84cf4a8-c298-4b59-8dfa-13c569bf8808'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.422275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cff24799eb24abcb8f7506a2ebc972a', 'request_id': 'req-e84cf4a8-c298-4b59-8dfa-13c569bf8808'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.436245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] The token cache is being invalidate because project 6cff24799eb24abcb8f7506a2ebc972a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:45.436245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.437445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.439455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1301/2616] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:45 2022] DELETE /identity/v3/projects/6cff24799eb24abcb8f7506a2ebc972a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:45.444141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:45.454331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095d40-4f51-457b-98c6-e2d043e3f149 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:45.511920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:45.512544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:45.512823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:45.513774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] PATH_INFO: `/v3/users/c3954dfd64de49d788a16f7d01043f64` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:45.514889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3954dfd64de49d788a16f7d01043f64)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:45.516836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:45.563341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3954dfd64de49d788a16f7d01043f64'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.570438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3954dfd64de49d788a16f7d01043f64'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.571108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3954dfd64de49d788a16f7d01043f64'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.579389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3954dfd64de49d788a16f7d01043f64'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.591498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ea8f1c63-c60e-4a3b-8920-97cea3f48cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3954dfd64de49d788a16f7d01043f64'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:45.597358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1316/2617] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:45 2022] DELETE /identity/v3/users/c3954dfd64de49d788a16f7d01043f64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:28:45.874137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:45.875667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:45.891832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf4a8-c298-4b59-8dfa-13c569bf8808 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:45.949184 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:45.949815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:45.950136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:45.950443 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] PATH_INFO: `/v3/projects/172b4dca6d9b41a7b581be9336d2f18f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:45.951594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172b4dca6d9b41a7b581be9336d2f18f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:45.953494 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.004415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172b4dca6d9b41a7b581be9336d2f18f', 'request_id': 'req-bd3a11ce-4f91-455f-ace2-7f31d647cef6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:46.010859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172b4dca6d9b41a7b581be9336d2f18f', 'request_id': 'req-bd3a11ce-4f91-455f-ace2-7f31d647cef6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:46.016435 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172b4dca6d9b41a7b581be9336d2f18f', 'request_id': 'req-bd3a11ce-4f91-455f-ace2-7f31d647cef6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:46.024713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:46.030156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] The token cache is being invalidate because project 172b4dca6d9b41a7b581be9336d2f18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:46.030448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:46.031108 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bd3a11ce-4f91-455f-ace2-7f31d647cef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:46.032630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1302/2618] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:45 2022] DELETE /identity/v3/projects/172b4dca6d9b41a7b581be9336d2f18f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:28:46.039163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.039697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.039795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.063541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:46.064736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:46.094133 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:46.127454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1317/2619] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:46.136491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.136778 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.137052 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.160355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:46.160355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:46.168091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1303/2620] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:46.173141 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.183087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b14983-b984-4582-8c1c-f0397fdcb766 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.187994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.188597 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.188883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.189162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.189843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:28:46.190171 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.191839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.203697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1318/2621] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:28:46.208063 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.217860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456826a-d212-45e3-ae6f-f8986ee444a8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.222857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.223487 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.223769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.224382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.225035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.226932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.256324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1304/2622] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:46.260561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.270504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf71989-53bb-40ba-a3c3-6304fbc8852e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.327332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.327928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.328206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.328478 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.329097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.331021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.352396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1319/2623] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:46.357269 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.367685 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99271252-83d6-4592-9f62-91549da408f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.372385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.372973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.373258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.373535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.374173 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.375810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.388708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1305/2624] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:46.396449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.410326 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974973c6-bf0e-4fdc-8259-26cfd2041427 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.415128 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.415758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.416038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.416308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.416948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.418820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75ebab-5b25-4385-b785-2b7e820b59d4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.433466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1320/2625] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:28:46.436289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.451328 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24935-bf31-400a-8337-47196b420907 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.455913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.456593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.456870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.457144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] PATH_INFO: `/v3/projects/b3a727e1f33d4f0990b9b572f0f2cb08/users/c39b587331c6478c8930add1cee3213e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.472383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a727e1f33d4f0990b9b572f0f2cb08, user_id=c39b587331c6478c8930add1cee3213e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.475007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.487626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1306/2626] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:46 2022] PUT /identity/v3/projects/b3a727e1f33d4f0990b9b572f0f2cb08/users/c39b587331c6478c8930add1cee3213e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:28:46.497826 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.498114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.498390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.526904 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] MFA Rules not processed for user `c39b587331c6478c8930add1cee3213e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:46.528196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:46.593795 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1321/2627] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:46.598495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.662260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7ebf-6a0c-4177-8514-3ec6a66de31e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.666912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.667544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.667830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.668110 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.668754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.670643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.696834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1307/2628] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:46.701731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.718217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33934e5-85ef-473d-96bc-b649022f6a04 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.749296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:46.782800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.783474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.783762 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.784052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.784670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.786599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.808055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1322/2629] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:46 2022] POST /identity/v3/users => generated 307 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:46.812622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.822975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58cb3-e30c-406b-8b8a-65e83dc0475c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.828265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.828857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.829140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.829417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.830092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.831720 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.842186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1308/2630] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:46.846273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.856269 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d956b-ee7f-433c-8622-a692e88cda11 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.861164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.861846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.862131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.862407 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] PATH_INFO: `/v3/projects/01e67eaf7ef74e989fbedc3229505583/users/d107c360ae154ef19d204240ca2877d7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.877883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e67eaf7ef74e989fbedc3229505583, user_id=d107c360ae154ef19d204240ca2877d7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.880884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.894970 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1323/2631] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:46 2022] PUT /identity/v3/projects/01e67eaf7ef74e989fbedc3229505583/users/d107c360ae154ef19d204240ca2877d7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:46.899188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.909656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2515e59-b4dc-41fb-9644-47eed3d4e672 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:46.966342 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:46.966949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:46.967260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:46.967549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:46.968187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:46.969783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b499de8-3712-4212-b905-aa1ab7824928 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:46.979699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1309/2632] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:46.984092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:46.994658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580585e9-2309-4f16-a24a-651fda5d2a8f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:47.000356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:47.001007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:47.001290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:47.001581 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] PATH_INFO: `/v3/projects/01e67eaf7ef74e989fbedc3229505583/users/d107c360ae154ef19d204240ca2877d7/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:47.003497 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e67eaf7ef74e989fbedc3229505583, user_id=d107c360ae154ef19d204240ca2877d7, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:47.006534 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:47.017303 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1324/2633] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:46 2022] PUT /identity/v3/projects/01e67eaf7ef74e989fbedc3229505583/users/d107c360ae154ef19d204240ca2877d7/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:47.027287 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:47.027587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:47.027872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:47.036230 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:47.060709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] MFA Rules not processed for user `d107c360ae154ef19d204240ca2877d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:47.062140 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:47.132695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1310/2634] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:47 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:47.143968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:47.145697 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:47.204705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572e4e2-c667-467c-ace1-b4d11c7c02a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:47.208426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:47.209037 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:47.209323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:47.209601 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:47.218827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:47.220306 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:47.229772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1325/2635] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:47 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:47.308981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:47.319255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e1e86-5920-47ae-bbe1-edfa555dd8be None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:47.322846 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:47.323460 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:47.323739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:47.324010 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:47.383175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:47.384870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa5e55-07f6-4bb5-958d-9150eaf8c49e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:47.394089 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1311/2636] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:47 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:51.435123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:51.496637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fa8cd-b19c-4599-94a8-316a88b2acb6 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:51.501250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:51.501878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:51.502162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:51.502437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] PATH_INFO: `/v3/users/c39b587331c6478c8930add1cee3213e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:51.503604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39b587331c6478c8930add1cee3213e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:51.505814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:51.533969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:51.553362 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39b587331c6478c8930add1cee3213e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:51.560395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39b587331c6478c8930add1cee3213e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:51.561064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39b587331c6478c8930add1cee3213e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:51.568504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39b587331c6478c8930add1cee3213e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:51.580772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39b587331c6478c8930add1cee3213e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:51.586845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1326/2637] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:51 2022] DELETE /identity/v3/users/c39b587331c6478c8930add1cee3213e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:51.592042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:51.592556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:51.592659 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:51.735641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:51.736849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:51.797909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1312/2638] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:51.807457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:51.808865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:51.865735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12932-9966-4ec5-9a92-f87f8ed1b5ae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:51.869330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:51.869932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:51.870217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:51.870490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:51.879983 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:51.881417 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:51.891725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1327/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:52.164845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:52.166388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:52.179855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e70a2-9fe1-4665-acf4-540e9075e5c1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:52.184930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:52.185572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:52.185850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:52.186121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] PATH_INFO: `/v3/projects/b3a727e1f33d4f0990b9b572f0f2cb08` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:52.187272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a727e1f33d4f0990b9b572f0f2cb08)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:52.189186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:52.214772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:52.237497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a727e1f33d4f0990b9b572f0f2cb08', 'request_id': 'req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.243630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a727e1f33d4f0990b9b572f0f2cb08', 'request_id': 'req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.249725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a727e1f33d4f0990b9b572f0f2cb08', 'request_id': 'req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.262216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] The token cache is being invalidate because project b3a727e1f33d4f0990b9b572f0f2cb08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:52.262500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.263793 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.265378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1313/2640] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:52 2022] DELETE /identity/v3/projects/b3a727e1f33d4f0990b9b572f0f2cb08 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:52.269402 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:52.279334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f36f6-0b5c-471f-92e1-5aea2b471a2b service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:52.337067 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:52.337636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:52.337924 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:52.338204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] PATH_INFO: `/v3/users/d107c360ae154ef19d204240ca2877d7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:52.339411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d107c360ae154ef19d204240ca2877d7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:52.341395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:52.392828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd107c360ae154ef19d204240ca2877d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.394068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:52.400777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd107c360ae154ef19d204240ca2877d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.401862 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd107c360ae154ef19d204240ca2877d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.409089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd107c360ae154ef19d204240ca2877d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.421377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd23568c-9045-42e3-b54b-5ad677f21d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd107c360ae154ef19d204240ca2877d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.427199 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1328/2641] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:52 2022] DELETE /identity/v3/users/d107c360ae154ef19d204240ca2877d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:52.721639 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:52.723185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:52.738858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52ae7-bfbc-4123-a3dc-f9416a8092d1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:52.799865 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:52.800498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:52.800782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:52.801062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] PATH_INFO: `/v3/projects/01e67eaf7ef74e989fbedc3229505583` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:52.802192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e67eaf7ef74e989fbedc3229505583)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:52.804118 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:52.849619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e67eaf7ef74e989fbedc3229505583', 'request_id': 'req-df09e5b6-6f03-4823-8810-3dce5c1f82b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.855654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e67eaf7ef74e989fbedc3229505583', 'request_id': 'req-df09e5b6-6f03-4823-8810-3dce5c1f82b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.862332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e67eaf7ef74e989fbedc3229505583', 'request_id': 'req-df09e5b6-6f03-4823-8810-3dce5c1f82b6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.875099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] The token cache is being invalidate because project 01e67eaf7ef74e989fbedc3229505583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:52.875099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.875099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-df09e5b6-6f03-4823-8810-3dce5c1f82b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:52.877741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1314/2642] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:52 2022] DELETE /identity/v3/projects/01e67eaf7ef74e989fbedc3229505583 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:52.882925 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:52.883461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:52.883562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:52.903088 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:52.904818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:52.962445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1329/2643] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:52.970506 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:52.970798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:52.971078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:52.991069 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:52.992326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:53.000306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1315/2644] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:53.004477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.014462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7ca9e-d2ae-4f43-af86-efa38205372c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.019115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.019744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.020034 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.020316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.020964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:28:53.021279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.022909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.036374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1330/2645] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:28:53.040808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.052153 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820477-9bfe-471b-b8c6-d2d1b2cefb7b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.056842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.057432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.057709 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.057978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.058615 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.060596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.086823 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1316/2646] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:28:53.091746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.102073 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a893c45-4118-4122-b3f2-fbb0f54f078e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.159125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.159757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.160334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.160621 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.161263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.163188 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.185289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1331/2647] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:53.189343 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.199984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50b4aa-66af-48f4-aa53-24a4ce141da2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.205217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.205815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.206101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.206375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.207045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.208739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.219603 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1317/2648] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:53.223839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.233864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a45da-2a99-4104-b0fd-8ea7c05a9909 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.238400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.238986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.239287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.239563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.240203 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.241797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba108798-e06a-492f-a18d-84fb04951f19 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.252318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1332/2649] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:53.256698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.261927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:53.271206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504ce33-761e-4c19-a838-a9047c7dd0cb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.277504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.278161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.278952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.279250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] PATH_INFO: `/v3/projects/3d92564bddf44c85a5473990ce42b644/users/3844fdc1f9ee4725a404e02e5ab5c2b1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.293859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d92564bddf44c85a5473990ce42b644, user_id=3844fdc1f9ee4725a404e02e5ab5c2b1, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.296585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.309511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1318/2650] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:53 2022] PUT /identity/v3/projects/3d92564bddf44c85a5473990ce42b644/users/3844fdc1f9ee4725a404e02e5ab5c2b1/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:28:53.322162 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.322450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.322730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.352293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] MFA Rules not processed for user `3844fdc1f9ee4725a404e02e5ab5c2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:53.353413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:53.371604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:53.420367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1333/2651] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:53.424861 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.491145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375b17c-8378-454d-83a8-7c7c5b0f06e0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.495948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.496550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.496835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.497113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.497756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.499693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.527795 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1319/2652] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:28:53.532580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.544373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d181c-a8e4-481c-a966-2d43f39b3e0a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.602676 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.603201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.603531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.603820 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.604465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.606432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.628077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1334/2653] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:28:53.632290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.643022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96b426-26ee-478e-8c00-6413b2ed668d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.647757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.648353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.648638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.648916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.649563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.651175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.661389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1320/2654] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:53.665572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.675485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeb7a7-4ea5-4365-8227-9a6887b56058 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.680144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.680783 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.681065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.681339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] PATH_INFO: `/v3/projects/82276f7932044b15a42bb8d02a59fe06/users/a0f0694dba2f4d898b975bcdff936a2f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.696631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82276f7932044b15a42bb8d02a59fe06, user_id=a0f0694dba2f4d898b975bcdff936a2f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.699519 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.713879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1335/2655] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:53 2022] PUT /identity/v3/projects/82276f7932044b15a42bb8d02a59fe06/users/a0f0694dba2f4d898b975bcdff936a2f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:53.718011 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.727901 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a297c7-d10d-4c40-8acb-26809db080d1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.786032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.786660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.786938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.787212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.787872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.789439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2282a-0166-4776-a2a1-433c97458047 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.799170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1321/2656] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:53.803756 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.814867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f115d1-20ed-4e2f-b884-1d14386812b3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:53.819694 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:53.820365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.820632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.820912 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] PATH_INFO: `/v3/projects/82276f7932044b15a42bb8d02a59fe06/users/a0f0694dba2f4d898b975bcdff936a2f/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.822860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82276f7932044b15a42bb8d02a59fe06, user_id=a0f0694dba2f4d898b975bcdff936a2f, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:53.826117 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:53.836283 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1336/2657] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:53 2022] PUT /identity/v3/projects/82276f7932044b15a42bb8d02a59fe06/users/a0f0694dba2f4d898b975bcdff936a2f/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:53.847166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:53.847490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:53.847769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:53.878025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] MFA Rules not processed for user `a0f0694dba2f4d898b975bcdff936a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:53.879179 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:53.919704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:53.952464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1322/2658] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:53.963069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:53.964894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:54.022637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d02c8-19e9-4adb-8184-65e6b74a2988 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:54.026270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:54.026864 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:54.027443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:54.027722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:54.036560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:54.037984 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:54.048831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1337/2659] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:54.150696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:54.160504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d68d4-0658-427b-bbe0-7027d777a811 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:54.164061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:54.164649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:54.164944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:54.165213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:54.220951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:54.222376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23112fa4-056f-4e6d-92f3-16d1a27b7b8e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:54.231151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1323/2660] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:56.816131 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:56.834625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:56.885259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5024c-ff3f-4e22-9d7c-94869dfdfb0a service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:56.892274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:56.892274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:56.892274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:56.892274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] PATH_INFO: `/v3/users/3844fdc1f9ee4725a404e02e5ab5c2b1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:56.896010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3844fdc1f9ee4725a404e02e5ab5c2b1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:56.896010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:56.946633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3844fdc1f9ee4725a404e02e5ab5c2b1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:56.953672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3844fdc1f9ee4725a404e02e5ab5c2b1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:56.954424 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3844fdc1f9ee4725a404e02e5ab5c2b1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:56.961955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3844fdc1f9ee4725a404e02e5ab5c2b1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:56.974838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3844fdc1f9ee4725a404e02e5ab5c2b1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:56.980947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1338/2661] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:56 2022] DELETE /identity/v3/users/3844fdc1f9ee4725a404e02e5ab5c2b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:56.986072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:56.986607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:56.986719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:57.010129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:57.011408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:57.080165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1324/2662] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:57.089830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:57.091486 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:57.156915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50114ed-cde6-4ed9-be9f-77e032db543b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:57.161144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:57.161881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:57.162173 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:57.162451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:57.172669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:57.174110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:57.188105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1339/2663] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:57.455012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:57.456533 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:57.474579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaf887-83c7-4650-9b31-373e600deb93 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:57.479787 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:57.480425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:57.480739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:57.481071 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] PATH_INFO: `/v3/projects/3d92564bddf44c85a5473990ce42b644` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:57.482770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d92564bddf44c85a5473990ce42b644)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:57.484812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:57.532922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d92564bddf44c85a5473990ce42b644', 'request_id': 'req-b47bdfc9-436e-481f-8c98-d06b621fb3f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.539056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d92564bddf44c85a5473990ce42b644', 'request_id': 'req-b47bdfc9-436e-481f-8c98-d06b621fb3f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.545236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d92564bddf44c85a5473990ce42b644', 'request_id': 'req-b47bdfc9-436e-481f-8c98-d06b621fb3f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.548883 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:57.560449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] The token cache is being invalidate because project 3d92564bddf44c85a5473990ce42b644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:57.560965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.561719 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.563498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1325/2664] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:57 2022] DELETE /identity/v3/projects/3d92564bddf44c85a5473990ce42b644 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:57.567573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:57.577454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b0d1-5d16-43b1-82d8-2e80052c64c8 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:57.612136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:57.637195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:57.637822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:57.638106 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:57.638383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] PATH_INFO: `/v3/users/a0f0694dba2f4d898b975bcdff936a2f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:57.639550 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f0694dba2f4d898b975bcdff936a2f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:57.641484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:57.688705 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f0694dba2f4d898b975bcdff936a2f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.699164 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f0694dba2f4d898b975bcdff936a2f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.700377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f0694dba2f4d898b975bcdff936a2f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.707997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f0694dba2f4d898b975bcdff936a2f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.721267 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1fb38d1-c70f-42c6-93c0-3cf39754b59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f0694dba2f4d898b975bcdff936a2f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:57.727389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1340/2665] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:28:57 2022] DELETE /identity/v3/users/a0f0694dba2f4d898b975bcdff936a2f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:58.003974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.005832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:58.018656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bdfc9-436e-481f-8c98-d06b621fb3f3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.077040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.077786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.078132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.078497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] PATH_INFO: `/v3/projects/82276f7932044b15a42bb8d02a59fe06` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.079718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=82276f7932044b15a42bb8d02a59fe06)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.081721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.129379 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82276f7932044b15a42bb8d02a59fe06', 'request_id': 'req-9289777b-658d-4779-ac99-217a7d46f85f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:58.135804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82276f7932044b15a42bb8d02a59fe06', 'request_id': 'req-9289777b-658d-4779-ac99-217a7d46f85f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:58.141317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82276f7932044b15a42bb8d02a59fe06', 'request_id': 'req-9289777b-658d-4779-ac99-217a7d46f85f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:58.154250 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] The token cache is being invalidate because project 82276f7932044b15a42bb8d02a59fe06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:28:58.154544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:58.155182 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9289777b-658d-4779-ac99-217a7d46f85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:28:58.156743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1326/2666] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:28:58 2022] DELETE /identity/v3/projects/82276f7932044b15a42bb8d02a59fe06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:58.161573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.162042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.162168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.182148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:58.183387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:58.248527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1341/2667] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:58.258567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.258934 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.259139 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.280777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:58.282022 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:58.290046 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1327/2668] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:58.295071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.305880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dec39f-969d-4c85-924e-1be5bb0035a5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.310685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.311364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.311652 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.312060 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.312726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:28:58.313044 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.314653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.330655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1342/2669] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:28:58.331858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.346246 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279776f-f8c9-4029-839a-da8e50ca44a6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.355068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.355770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.356712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.357072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.357929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.359913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.390932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1328/2670] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:28:58 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:58.395143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.406312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd77ab-f0eb-46b7-a521-e821ccaae610 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.452880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:58.470205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.470744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.471029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.471339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.471982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.473938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.497114 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1343/2671] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:28:58 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:28:58.502779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.514042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977046a-3b73-40b2-bab1-77e7ef9aa75e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.518650 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.519271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.519556 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.519826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.520462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.522049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.532717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1329/2672] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:58.536740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.546608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf2af-23a2-41b4-bb01-28c42bd5af33 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.551295 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.551896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.552177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.552451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.553122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.554710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4b3c-5b14-454b-9dca-48697472dfbf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.565329 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1344/2673] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:28:58.569449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.579903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6226a-db74-4d0e-ada9-19357f59435c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.585068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.585753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.586037 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.586339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] PATH_INFO: `/v3/projects/7c7226670c9c45139ae6ed0c6c8684de/users/0cf872f02e8e4b4bba6ffc91f0c58d6e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.601873 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7226670c9c45139ae6ed0c6c8684de, user_id=0cf872f02e8e4b4bba6ffc91f0c58d6e, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.604555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.617399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1330/2674] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:28:58 2022] PUT /identity/v3/projects/7c7226670c9c45139ae6ed0c6c8684de/users/0cf872f02e8e4b4bba6ffc91f0c58d6e/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:28:58.627928 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.628382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.628542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.660748 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] MFA Rules not processed for user `0cf872f02e8e4b4bba6ffc91f0c58d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:28:58.662055 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:58.728050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1345/2675] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:28:58.738840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:58.740879 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:58.771254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:28:58.805219 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ebe2-08e2-45d7-8b3b-952bde204438 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:58.809068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:58.810332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:58.810614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:58.810886 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:58.819429 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:58.820888 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:58.830331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1331/2676] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:28:59.512462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:28:59.514582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:59.635736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73858d1-d2d0-4d24-b0b8-90e8a483fabf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:28:59.645635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:28:59.646334 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:28:59.646619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:28:59.646896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:28:59.648754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:28:59.664567 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:28:59.682482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea0cd9-1d0c-47fa-af82-22e4bef474e9 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:28:59.703847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1346/2677] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:07.293402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:07.359061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af0f63-7021-4848-9851-58f3bae3764f service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:07.364054 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:07.364684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:07.364963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:07.365235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] PATH_INFO: `/v3/users/0cf872f02e8e4b4bba6ffc91f0c58d6e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:07.366356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf872f02e8e4b4bba6ffc91f0c58d6e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:07.368513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:07.425656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf872f02e8e4b4bba6ffc91f0c58d6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:07.433988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf872f02e8e4b4bba6ffc91f0c58d6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:07.434711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf872f02e8e4b4bba6ffc91f0c58d6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:07.435386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf872f02e8e4b4bba6ffc91f0c58d6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:07.444785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf872f02e8e4b4bba6ffc91f0c58d6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:07.470437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1332/2678] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:29:07 2022] DELETE /identity/v3/users/0cf872f02e8e4b4bba6ffc91f0c58d6e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:07.474963 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:07.475530 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:07.475671 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:07.497193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:07.499193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:07.540726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:07.572668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1347/2679] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:29:07.582663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:07.584241 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:07.635056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:07.647922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e4cf-acab-424e-9e09-f1b8fc6e6758 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:07.653098 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:07.653336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:07.653619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:07.653893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:07.663399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:07.664831 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713abe-c4a2-4b9a-bf56-f0e8e163fe50 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:07.675881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1333/2680] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:08.023731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.026421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:08.039520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fad58-013d-4d08-ac74-595f57544125 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.044526 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.045839 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.046223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.047274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] PATH_INFO: `/v3/projects/7c7226670c9c45139ae6ed0c6c8684de` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.048715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c7226670c9c45139ae6ed0c6c8684de)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.050953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.114566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c7226670c9c45139ae6ed0c6c8684de', 'request_id': 'req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:08.122014 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c7226670c9c45139ae6ed0c6c8684de', 'request_id': 'req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:08.129005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c7226670c9c45139ae6ed0c6c8684de', 'request_id': 'req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:08.140460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] The token cache is being invalidate because project 7c7226670c9c45139ae6ed0c6c8684de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:29:08.140761 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:08.141474 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-033fbbe3-2251-4c8d-bcf3-fea31a9bceb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:08.148064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1348/2681] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:29:08 2022] DELETE /identity/v3/projects/7c7226670c9c45139ae6ed0c6c8684de => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:08.149762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.150057 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.150341 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.172373 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:08.173605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:08.246159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1334/2682] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:08.312693 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.312957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.313213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.335418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:08.336653 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:08.345632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1349/2683] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:08.350383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.361387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9579105-f984-41ca-8a65-d29b034465e6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.366439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.367042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.367351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.367623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.368266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:29:08.368575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.370151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.382348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1335/2684] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:29:08.386712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.396948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756c00-f090-44ff-9260-ca61b7d739f4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.402029 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.402742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.403091 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.403455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.404223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.407285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.443884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1350/2685] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:08.448001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.458815 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f804-3b42-4ad0-96b1-25742e82bc8a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.519586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.520280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.520627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.520902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.521581 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.523549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.547336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1336/2686] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:29:08.551473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.564094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d28d83-f52f-4f62-885b-2c88a6c24b9e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.568730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.569343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.569631 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.570122 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.570874 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.572523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.584953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1351/2687] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:08.589196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.600412 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5db43-b287-46be-beed-021cd2b52d92 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.605591 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.606208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.606496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.607198 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.607456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.609043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a3b51-e6ac-44b2-a240-4d6eddc777d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.624969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1337/2688] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:29:08.626197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.637755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d63e59-9f88-424c-be85-0f189754bd72 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.646906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.646906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.646906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.646906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] PATH_INFO: `/v3/projects/e1f708cad9e843c69781bc702eea1e4a/users/697eadee87d84424bca3b3d0c11f695d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.664849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f708cad9e843c69781bc702eea1e4a, user_id=697eadee87d84424bca3b3d0c11f695d, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.668943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.684913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1352/2689] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:08 2022] PUT /identity/v3/projects/e1f708cad9e843c69781bc702eea1e4a/users/697eadee87d84424bca3b3d0c11f695d/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:08.695344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.695718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.696045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.728388 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] MFA Rules not processed for user `697eadee87d84424bca3b3d0c11f695d`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:08.731634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:08.779545 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:08.812451 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1338/2690] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:08.817261 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.912560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:08.916512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a20ec-08a0-4787-baa3-8084ad4a2f1a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:08.927669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:08.928297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:08.928583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:08.928887 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:08.929547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:08.931658 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:08.959725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1353/2691] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:29:08.963926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:08.975032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06dbd2-a71c-4092-a066-8d8f99b0875c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.033614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.034229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.034511 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.034788 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.035462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.037392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.062670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1339/2692] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:29:08 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:09.066655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.077077 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d871d7-c230-48fa-a418-dbfcd7c0cb0a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.082249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.082854 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.083138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.083444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.084093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.085730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.096941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1354/2693] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:09.101125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.112965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085b6a-e06a-455d-a050-4eca4ebcc9b1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.117915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.118524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.118806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.119082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.119766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.121372 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc56d-45fb-473e-b9b5-601f7cea4fb0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.132767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1340/2694] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:29:09.137094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.147445 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115224ff-c274-4f89-aa88-cd2d338d3d05 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.152147 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.152799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.153084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.153361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] PATH_INFO: `/v3/projects/f61370a79c164fb28fdce887481e8132/users/ce1b8f8d551d47fa924b3b49cd2a772a/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.169208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61370a79c164fb28fdce887481e8132, user_id=ce1b8f8d551d47fa924b3b49cd2a772a, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.171976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.187264 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1355/2695] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:09 2022] PUT /identity/v3/projects/f61370a79c164fb28fdce887481e8132/users/ce1b8f8d551d47fa924b3b49cd2a772a/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:09.198620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.198995 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.199387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.230213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] MFA Rules not processed for user `ce1b8f8d551d47fa924b3b49cd2a772a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:09.231767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:09.309406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1341/2696] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:09.311280 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.380248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bada71-7b3b-4622-8d71-dcf7911099de admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.385699 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.386312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.386929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.386929 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.387549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.389834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.425149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1356/2697] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:29:09 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:09.426294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.437481 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb48120-6d6a-4a1a-8d32-46c97038b3ef None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.492074 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:09.504245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.505356 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.505714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.506101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.507122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.509310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.533209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1342/2698] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:29:09 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:29:09.537391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.554805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797767-7ee1-45d2-9503-a9fa65341c12 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.562607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.563111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.563425 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.563739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.564565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.566385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.578479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1357/2699] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:09.582817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.593477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b6a2c-6658-48e1-8f0a-36e4646277a2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.600210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.600900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.601211 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.601495 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] PATH_INFO: `/v3/projects/7a950c453c824c77907c9a59727868f1/users/061d54c6fa344efca7dc5c88c90d2485/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.623811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a950c453c824c77907c9a59727868f1, user_id=061d54c6fa344efca7dc5c88c90d2485, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.627745 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.650850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1343/2700] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:09 2022] PUT /identity/v3/projects/7a950c453c824c77907c9a59727868f1/users/061d54c6fa344efca7dc5c88c90d2485/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:09.655410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.668490 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba870af-8531-4df0-a2bc-78b7b3cfb715 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.741317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.741955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.742239 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.742520 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.743273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.744936 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e01cd-7832-47f7-85b2-785ecc753287 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.755856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1358/2701] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:29:09.760213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.772081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732aa3-c0c3-4a18-9ff2-19455f70889c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:09.777145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:09.777827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.778235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.778411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] PATH_INFO: `/v3/projects/7a950c453c824c77907c9a59727868f1/users/061d54c6fa344efca7dc5c88c90d2485/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.781216 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a950c453c824c77907c9a59727868f1, user_id=061d54c6fa344efca7dc5c88c90d2485, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:09.784024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:09.795969 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1344/2702] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:09 2022] PUT /identity/v3/projects/7a950c453c824c77907c9a59727868f1/users/061d54c6fa344efca7dc5c88c90d2485/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:09.806548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61db34-e4d9-4316-98f5-abfff4463211 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:09.806838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61db34-e4d9-4316-98f5-abfff4463211 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:09.807116 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61db34-e4d9-4316-98f5-abfff4463211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:09.839332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b61db34-e4d9-4316-98f5-abfff4463211 None None] MFA Rules not processed for user `061d54c6fa344efca7dc5c88c90d2485`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:09.844108 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b61db34-e4d9-4316-98f5-abfff4463211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:09.847305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:09.921266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1359/2703] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:09.932884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:09.934630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:09.995872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0dc31-b6b6-4965-bb95-e404662a2796 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:10.000466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:10.001085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:10.001371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:10.001654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:10.064680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:10.066149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:10.075497 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1345/2704] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:10.683894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:10.684516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:10.684516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:10.704709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] MFA Rules not processed for user `ea15538623ef4bacb42c4e03e2db72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:10.705960 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:10.766414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1360/2705] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:11.438101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:11.439925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:11.515643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:11.531265 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca131-e438-4efc-bdb4-65c537de5411 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:11.536264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:11.536874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:11.537161 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:11.537596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:11.539151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:11.551441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:11.553090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:11.564263 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1346/2706] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:11.646664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:11.648812 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:11.661273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf2343-b4e2-4fc8-8fb9-825e8552bff6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:11.665307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:11.665920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:11.666201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:11.666469 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:11.727686 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:11.729218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:11.738753 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1361/2707] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:12.322950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:12.332254 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0866c39-bb8b-47a7-80ca-2a14398144d8 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:12.335906 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:12.336522 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:12.336806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:12.337078 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:12.346386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:12.347839 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d9f8c-c406-4b16-957e-67a714b3dc99 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:12.357994 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1347/2708] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:16.269851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:16.317487 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:16.333130 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b487117-56dc-4b93-9879-a4df86a46782 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:16.337618 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:16.338265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:16.338545 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:16.338821 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] PATH_INFO: `/v3/users/697eadee87d84424bca3b3d0c11f695d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:16.339990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] RBAC: Authorizing `identity:delete_user(user_id=697eadee87d84424bca3b3d0c11f695d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:16.341915 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:16.406226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697eadee87d84424bca3b3d0c11f695d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:16.414003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697eadee87d84424bca3b3d0c11f695d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:16.415127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697eadee87d84424bca3b3d0c11f695d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:16.424223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697eadee87d84424bca3b3d0c11f695d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:16.437640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697eadee87d84424bca3b3d0c11f695d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:16.444574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1362/2709] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:29:16 2022] DELETE /identity/v3/users/697eadee87d84424bca3b3d0c11f695d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:16.451404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:16.451924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:16.452208 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:16.470959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:16.472185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:16.537438 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1348/2710] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:16.547505 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:16.548938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:16.610121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528e26-67b3-4d88-9212-12106bef542f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:16.613901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:16.614515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:16.614802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:16.615105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:16.624590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:16.626050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:16.636774 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1363/2711] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:17.244850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:17.246290 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:17.261164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58f08b-9a83-4a84-8e8a-147f76619c1b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:17.267121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:17.267804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:17.268275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:17.268564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] PATH_INFO: `/v3/projects/e1f708cad9e843c69781bc702eea1e4a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:17.269722 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f708cad9e843c69781bc702eea1e4a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:17.271928 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:17.329876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f708cad9e843c69781bc702eea1e4a', 'request_id': 'req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.336573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f708cad9e843c69781bc702eea1e4a', 'request_id': 'req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.342167 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f708cad9e843c69781bc702eea1e4a', 'request_id': 'req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.354716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] The token cache is being invalidate because project e1f708cad9e843c69781bc702eea1e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:29:17.355018 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.355654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.357938 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1349/2712] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:29:17 2022] DELETE /identity/v3/projects/e1f708cad9e843c69781bc702eea1e4a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:17.361738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:17.372337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b551db5-2fd8-4c1d-9ca0-5419c8322b3a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:17.436161 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:17.436800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:17.437085 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:17.437395 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] PATH_INFO: `/v3/users/ce1b8f8d551d47fa924b3b49cd2a772a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:17.438758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1b8f8d551d47fa924b3b49cd2a772a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:17.441410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:17.452136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:17.489323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1b8f8d551d47fa924b3b49cd2a772a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.496845 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1b8f8d551d47fa924b3b49cd2a772a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.498095 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1b8f8d551d47fa924b3b49cd2a772a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.506308 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1b8f8d551d47fa924b3b49cd2a772a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.519716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1b8f8d551d47fa924b3b49cd2a772a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.525739 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1364/2713] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:29:17 2022] DELETE /identity/v3/users/ce1b8f8d551d47fa924b3b49cd2a772a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:17.827811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:17.829340 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:17.845671 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af7b-9f4d-4c98-a33c-a13ef9db4e9e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:17.902580 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:17.910463 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:17.911144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:17.911937 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:17.912279 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] PATH_INFO: `/v3/projects/f61370a79c164fb28fdce887481e8132` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:17.914106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61370a79c164fb28fdce887481e8132)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:17.916271 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:17.971034 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61370a79c164fb28fdce887481e8132', 'request_id': 'req-e8da92c6-0c67-4af8-8138-19cafc018eb9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.977832 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61370a79c164fb28fdce887481e8132', 'request_id': 'req-e8da92c6-0c67-4af8-8138-19cafc018eb9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.984378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61370a79c164fb28fdce887481e8132', 'request_id': 'req-e8da92c6-0c67-4af8-8138-19cafc018eb9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.997838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] The token cache is being invalidate because project f61370a79c164fb28fdce887481e8132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:29:17.998132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:17.998777 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.000424 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1350/2714] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:29:17 2022] DELETE /identity/v3/projects/f61370a79c164fb28fdce887481e8132 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:18.004769 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.006361 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:18.020206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d4e71-a843-49a4-9e0b-e941f29cc69e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.084319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.084958 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.085253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.085521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] PATH_INFO: `/v3/users/061d54c6fa344efca7dc5c88c90d2485` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.086644 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=061d54c6fa344efca7dc5c88c90d2485)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:18.088623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:18.138039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '061d54c6fa344efca7dc5c88c90d2485'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.145712 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '061d54c6fa344efca7dc5c88c90d2485'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.146410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '061d54c6fa344efca7dc5c88c90d2485'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.155948 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '061d54c6fa344efca7dc5c88c90d2485'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.169144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe08ef27-541e-4018-881f-51f47745c088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '061d54c6fa344efca7dc5c88c90d2485'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.175342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1365/2715] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:29:18 2022] DELETE /identity/v3/users/061d54c6fa344efca7dc5c88c90d2485 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:18.460932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.462645 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:18.484691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da92c6-0c67-4af8-8138-19cafc018eb9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.558421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.559077 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.559453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.559802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] PATH_INFO: `/v3/projects/7a950c453c824c77907c9a59727868f1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.561643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a950c453c824c77907c9a59727868f1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:18.563941 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:18.632854 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a950c453c824c77907c9a59727868f1', 'request_id': 'req-a7bd808e-b208-4d89-933a-c37565418f47'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.640472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a950c453c824c77907c9a59727868f1', 'request_id': 'req-a7bd808e-b208-4d89-933a-c37565418f47'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.650908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a950c453c824c77907c9a59727868f1', 'request_id': 'req-a7bd808e-b208-4d89-933a-c37565418f47'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.664549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] The token cache is being invalidate because project 7a950c453c824c77907c9a59727868f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:29:18.664852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.665539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a7bd808e-b208-4d89-933a-c37565418f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:29:18.667092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1351/2716] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:29:18 2022] DELETE /identity/v3/projects/7a950c453c824c77907c9a59727868f1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:18.672643 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.673166 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.673268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.687541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:18.694113 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:18.695884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:18.756270 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1366/2717] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:18.764414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.764693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.765001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.784264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:18.785405 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:18.793667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1352/2718] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:18.798574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.808415 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885231ef-66fe-4c25-bf15-15a063c36003 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.813151 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.813692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.813972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.814248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.814890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:29:18.815195 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:18.816875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:18.829072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1367/2719] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:29:18.833394 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.843399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be893e-7ce1-49d9-b49c-3bd6f22d6f1f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.848090 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.848687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.848991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.849266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.850007 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:18.851932 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:18.861488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:18.881458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1353/2720] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:29:18 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:18.886031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.896670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521d17f-b9f6-4d7b-8372-3903eb68f832 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.952380 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.952979 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.953260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.953532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.954179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:18.956124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:18.977274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1368/2721] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:29:18 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:18.981368 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:18.991713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e4107-7666-4590-881c-a11438f139a7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:18.997217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:18.998307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:18.998590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:18.998868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:18.999544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:19.001144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:19.011808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1354/2722] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:19.016358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:19.027413 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bbf84-34f9-43a9-a22d-6cc760aef18e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:19.032125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:19.032730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:19.033011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:19.033285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:19.033930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:19.035589 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e5044-ca15-4543-a72d-8a77e05257d6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:19.046608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1369/2723] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:19.051397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:19.062779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7e149-a073-48f1-971e-3a16610b2acc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:19.067605 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:19.068187 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:19.068468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:19.068739 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] PATH_INFO: `/v3/projects/63e7ee5c71674c23aca48909b417a52a/users/20cf8769967d4169be2a2a82431c7df6/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:19.084028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e7ee5c71674c23aca48909b417a52a, user_id=20cf8769967d4169be2a2a82431c7df6, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:19.086649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:19.099566 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1355/2724] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:19 2022] PUT /identity/v3/projects/63e7ee5c71674c23aca48909b417a52a/users/20cf8769967d4169be2a2a82431c7df6/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:19.104583 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:19.104877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:19.105157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:19.124452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:19.125662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:19.186455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1370/2725] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:19.196874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:19.198614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:19.255755 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e7f6e-1ea3-4a60-b73f-e5cd85316733 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:19.259448 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:19.260050 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:19.260334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:19.260611 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:19.270188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:19.271656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77819ae5-00d7-4881-a29c-4418aac736b1 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:19.281245 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1356/2726] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:20.101242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:20.104043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:20.127588 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:20.213363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5d50f-d149-40ec-9245-b834cbf4794b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:20.220319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:20.220614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:20.220901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:20.221176 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:20.223369 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:20.237863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:20.239522 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:20.254094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1371/2727] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:29:23.712721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:23.713262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:23.714129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:23.761274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] MFA Rules not processed for user `20cf8769967d4169be2a2a82431c7df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:23.762643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:23.908770 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:23.909335 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1357/2728] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:23.927701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e151-a6fe-4f0c-a797-c364ad02449f service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:23.932817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:23.933443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:23.933728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:23.934005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:23.934655 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:23.937179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.013809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1372/2729] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:29:23 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:24.015395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:24.033653 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dea998-b85e-4fd8-a72d-7ed12af13315 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:24.109496 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:24.141524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:24.142943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:24.142943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:24.142943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:24.143899 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:24.146520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.188714 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1358/2730] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:29:24 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:29:24.194401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:24.216396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40842b4-f6dc-4eca-999a-dd032bd7f2a9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:24.223800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:24.224747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:24.225144 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:24.227537 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:24.228580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:24.230793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.250523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1373/2731] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:24.256027 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:24.268898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078599d-b66d-4e03-8221-ab31a9434b2a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:24.277891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:24.279696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:24.280275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:24.280759 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] PATH_INFO: `/v3/projects/8bc96483ba614f5a8b1b71f53292a240/users/840fc526a7fa4831835b4757e8ee1e38/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:24.313199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc96483ba614f5a8b1b71f53292a240, user_id=840fc526a7fa4831835b4757e8ee1e38, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:24.321253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.355503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1359/2732] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:24 2022] PUT /identity/v3/projects/8bc96483ba614f5a8b1b71f53292a240/users/840fc526a7fa4831835b4757e8ee1e38/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:29:24.357639 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:24.370742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082908-271d-4de9-97f1-fc8fa1ea89e4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:24.497815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:24.498450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:24.498734 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:24.499010 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:24.499714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:24.501342 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999328d3-5889-4802-8c95-7c5757811c30 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.515461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1374/2733] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:29:24.517205 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:24.532428 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0509484-8478-4307-9cd1-39d763187526 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:24.537610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:24.538524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:24.538961 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:24.539721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] PATH_INFO: `/v3/projects/8bc96483ba614f5a8b1b71f53292a240/users/840fc526a7fa4831835b4757e8ee1e38/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:24.542585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc96483ba614f5a8b1b71f53292a240, user_id=840fc526a7fa4831835b4757e8ee1e38, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:24.546442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:24.558827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1360/2734] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:29:24 2022] PUT /identity/v3/projects/8bc96483ba614f5a8b1b71f53292a240/users/840fc526a7fa4831835b4757e8ee1e38/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:29:28.947292 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167f8dd-fb7a-49e9-8cbd-d39e36da42db None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:28.947824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167f8dd-fb7a-49e9-8cbd-d39e36da42db None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:28.947824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167f8dd-fb7a-49e9-8cbd-d39e36da42db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:28.982511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5167f8dd-fb7a-49e9-8cbd-d39e36da42db None None] MFA Rules not processed for user `840fc526a7fa4831835b4757e8ee1e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:28.984265 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5167f8dd-fb7a-49e9-8cbd-d39e36da42db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:29.043129 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:29.093179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1375/2735] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:29:29.109383 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:29:29.111310 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:29.196491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1e181-7526-4e85-ba93-4727a1e13eec admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:29:29.203617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:29:29.204240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:29.204548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:29.204830 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:29.268318 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:29:29.269929 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:29:29.279540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1361/2736] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:29:39.089401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:39.274811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:29:52.976609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:29:52.977157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:29:52.977157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:29:52.996026 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] MFA Rules not processed for user `20cf8769967d4169be2a2a82431c7df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:29:52.997227 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:29:53.004234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1376/2737] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:30:02.990648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:30:10.708387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:30:10.775047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd179d6-6e4e-41e2-a6da-73413f4163bc service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:30:10.779083 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:30:10.779750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:10.780072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:10.780350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:10.790786 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:30:10.792280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:30:10.803386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1362/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:30:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:30:12.436691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:30:12.451206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc040d7-d2cc-4b51-8f2c-9e094306a761 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:30:12.457027 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:30:12.457648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:12.457930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:12.458202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:12.460066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:12.473148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:30:12.474731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:30:12.486028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1377/2739] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:30:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:30:20.798942 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=26 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:30:22.480910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:30:40.017525 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:30:40.029723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2666c-be8c-4fa1-8127-545eeb5d91f1 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:30:40.035875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:30:40.036462 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:40.036743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:40.037016 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:40.038743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:40.097378 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:30:40.098975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69a844-802e-4863-911a-315281b1c555 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:30:40.108350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1363/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:30:50.103930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=28 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:30:53.905850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:30:53.908124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:53.985245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914720-a947-4260-9d6a-babddf697a13 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:30:53.991883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:30:53.992802 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:53.993879 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:53.994112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:53.996552 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:54.011910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:30:54.013541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:30:54.024969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1378/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:30:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:30:57.527535 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:57.528096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:57.528096 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:57.547769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] MFA Rules not processed for user `ea15538623ef4bacb42c4e03e2db72c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:30:57.549061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:57.556734 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1364/2742] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:30:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:30:57.863836 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:30:57.865234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:57.876672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75e0b4-48ba-4cfa-80d7-0b1ada66b027 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:30:57.880527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:30:57.881123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:30:57.881400 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:30:57.881672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:30:57.883462 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:30:57.893781 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:30:57.895213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:30:57.902999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1379/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:30:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:31:07.542210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:31:07.899523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=38 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:16.268783 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:16.270221 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:16.326091 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724efda-c9a0-40b0-85e4-90a123201c4e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:16.329687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:16.330282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:16.330562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:16.330838 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:16.332612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:16.345673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:16.347092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df8981-7a9a-47dc-880b-08ae937c62e7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:16.357843 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1365/2744] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:24.234720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:24.236208 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:24.248138 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73b52f-2d08-4781-ab1e-9fada4881941 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:24.251720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:24.252317 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:24.252599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:24.252870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:24.254805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:24.267962 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:24.269388 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:24.280406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1380/2745] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:32:26.353166 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=26 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:33.298756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:33.299756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:33.299756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:33.318676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] MFA Rules not processed for user `20cf8769967d4169be2a2a82431c7df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:33.319912 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:33.326701 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1366/2746] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:32:33.911025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:33.920336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8673a8c-5cc5-4541-b207-3cab75fac275 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:33.924049 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:33.924489 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:33.924775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:33.925051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:33.926755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:33.998249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:33.999747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:34.011387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1381/2747] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:37.591393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:37.611563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bf0d-caf7-4b54-af11-fbd0e9787913 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:37.617367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:37.617970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:37.618470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:37.618914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:37.620776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:37.636389 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:37.637986 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:37.651808 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1367/2748] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:37.676457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:37.678523 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:37.690775 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16329-d278-410d-a06b-652d1724976a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:37.695494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:37.696757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:37.696757 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:37.698350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] PATH_INFO: `/v3/users/ea15538623ef4bacb42c4e03e2db72c5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:37.699668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea15538623ef4bacb42c4e03e2db72c5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:37.701623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:37.763665 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea15538623ef4bacb42c4e03e2db72c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:37.771052 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea15538623ef4bacb42c4e03e2db72c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:37.771726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea15538623ef4bacb42c4e03e2db72c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:37.780989 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea15538623ef4bacb42c4e03e2db72c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:37.794529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f1e06dd-19ec-4169-bbba-9b2875052fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea15538623ef4bacb42c4e03e2db72c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:37.800990 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1382/2749] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:32:37 2022] DELETE /identity/v3/users/ea15538623ef4bacb42c4e03e2db72c5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:37.980274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:37.992913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6ee6b-52fa-4fb9-9ee8-8b1444a442cc service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.081555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.082202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.082488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.083439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] PATH_INFO: `/v3/projects/d08b7bad519b4188b8cef19735f023fe` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.084629 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08b7bad519b4188b8cef19735f023fe)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.086577 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.136045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08b7bad519b4188b8cef19735f023fe', 'request_id': 'req-fc028f70-f670-4019-a4c7-d9b1e93e3cab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:38.143314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08b7bad519b4188b8cef19735f023fe', 'request_id': 'req-fc028f70-f670-4019-a4c7-d9b1e93e3cab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:38.150160 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08b7bad519b4188b8cef19735f023fe', 'request_id': 'req-fc028f70-f670-4019-a4c7-d9b1e93e3cab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:38.164845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] The token cache is being invalidate because project d08b7bad519b4188b8cef19735f023fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:32:38.165137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:38.165811 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc028f70-f670-4019-a4c7-d9b1e93e3cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:38.167583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1368/2750] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:32:37 2022] DELETE /identity/v3/projects/d08b7bad519b4188b8cef19735f023fe => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:38.172323 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.172873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.173149 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.194125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:38.195357 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:38.252967 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:38.269852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1383/2751] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:38.278385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.278633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.278884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.300992 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:38.302427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:38.310488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1369/2752] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:38.314743 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.325102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeb4dc-c477-4233-891b-9469f0610a13 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.330136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.330690 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.330964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.331263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.331927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:32:38.332226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.334685 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.347442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1384/2753] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:32:38.351923 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.361260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631916f-4d4f-464d-aff6-732b34128a04 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.366017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.366619 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.366900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.367172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.367847 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.369785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.399706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1370/2754] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:38.404507 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.423178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ec952-a706-4c3b-b0c0-53b2d8191eca admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.509007 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.509729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.511102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.511102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.511102 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.513136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.540480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1385/2755] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:38.545909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.561092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aeff68-9674-41fd-8a32-4ad7727a8535 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.567903 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.568951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.569423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.569866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.570859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.573002 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.595695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1371/2756] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:32:38.598881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.611187 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ce1b-8f5f-43db-b02c-75d28eb27c21 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.618033 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.618876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.619408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.619907 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.621063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.623275 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1c69c-ae28-4048-86c0-5f76187edd8a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.636202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1386/2757] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:32:38.640757 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.652286 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401635-e210-4a3c-aa2d-080532543ea5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.657455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.658121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.658408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.658688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] PATH_INFO: `/v3/projects/2ca815709d0b4d9182c17172471f6389/users/291ac3554bb8415098c5df735b728efd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.673561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca815709d0b4d9182c17172471f6389, user_id=291ac3554bb8415098c5df735b728efd, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.676281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.689840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1372/2758] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:38 2022] PUT /identity/v3/projects/2ca815709d0b4d9182c17172471f6389/users/291ac3554bb8415098c5df735b728efd/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:38.701818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.702377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.702377 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.732455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] MFA Rules not processed for user `291ac3554bb8415098c5df735b728efd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:38.733696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:38.800251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1387/2759] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:38.805262 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.814596 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:38.873532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f106-58d6-49d4-b57c-04efd06946d4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.878850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:38.879058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:38.879322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:38.880441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:38.881122 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:38.882987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:38.910572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1373/2760] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:38.916284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:38.928081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcb85c-1c2b-403b-9bc8-dd651164896d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:38.979246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:39.000043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.000650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.000931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.001200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.001825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.003823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.029193 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1388/2761] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:32:38 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:39.037924 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.048613 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195a397-de60-4cab-847d-c5964e3495b8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.055666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.056209 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.056491 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.056768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.057508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.059190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.072105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1374/2762] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:32:39.076472 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.086827 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72138d5-04e0-41e8-8385-223a396e7831 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.091561 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.092226 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.092509 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.092786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] PATH_INFO: `/v3/projects/bb29be4c5b3746989e1fda0b71a93d77/users/7f49766b24404d7e97864622353df68b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.108364 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb29be4c5b3746989e1fda0b71a93d77, user_id=7f49766b24404d7e97864622353df68b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.111064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.124942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1389/2763] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:39 2022] PUT /identity/v3/projects/bb29be4c5b3746989e1fda0b71a93d77/users/7f49766b24404d7e97864622353df68b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:39.129020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.139244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d300c-b392-4274-9d78-eae5b5c9f2e4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.195698 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.196311 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.196589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.197072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.197500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.199084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975b248-7710-4037-a6a7-4c89074aac52 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.209255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1375/2764] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:32:39.213414 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.224482 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacee24-215b-457e-a5a5-ce145dde6b39 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.229746 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.230392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.230696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.230969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] PATH_INFO: `/v3/projects/bb29be4c5b3746989e1fda0b71a93d77/users/7f49766b24404d7e97864622353df68b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.232953 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb29be4c5b3746989e1fda0b71a93d77, user_id=7f49766b24404d7e97864622353df68b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.235636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.247064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1390/2765] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:39 2022] PUT /identity/v3/projects/bb29be4c5b3746989e1fda0b71a93d77/users/7f49766b24404d7e97864622353df68b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:39.257530 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.257814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.258119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.286260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] MFA Rules not processed for user `7f49766b24404d7e97864622353df68b`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:39.287500 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:39.355587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1376/2766] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:39.366877 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.368687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:39.434615 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27d32-fda6-4501-8084-be35ea790d87 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.438279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.438880 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.439160 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.439456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.448340 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.449773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7336-0af1-47d5-8263-493bfbbdff16 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:39.459464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1391/2767] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:39.895020 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:39.899974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:39.959555 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14cb47-d1ce-403a-a764-3bf10006d536 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:39.964065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:39.964687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:39.964974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:39.965252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] PATH_INFO: `/v3/users/291ac3554bb8415098c5df735b728efd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:39.966371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=291ac3554bb8415098c5df735b728efd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:39.968316 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:40.017910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291ac3554bb8415098c5df735b728efd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.025979 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291ac3554bb8415098c5df735b728efd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.026675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291ac3554bb8415098c5df735b728efd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.027322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291ac3554bb8415098c5df735b728efd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.035547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291ac3554bb8415098c5df735b728efd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.057035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1377/2768] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:32:39 2022] DELETE /identity/v3/users/291ac3554bb8415098c5df735b728efd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:32:40.061614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:40.062178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:40.062178 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:40.084885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:40.085946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:40.273634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1392/2769] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:40.283267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:40.284676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:40.340699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08464-8db6-4a3c-950e-e8eb3e4379cb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:40.344233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:40.344827 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:40.345104 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:40.345376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:40.354212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:40.355667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:40.365655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1378/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:40.645145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:40.646670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:40.658725 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c93df-feab-4a53-8ac3-1fae72a19a2c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:40.663856 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:40.664511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:40.664816 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:40.665084 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] PATH_INFO: `/v3/projects/2ca815709d0b4d9182c17172471f6389` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:40.666285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca815709d0b4d9182c17172471f6389)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:40.668245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:40.709732 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:40.718293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca815709d0b4d9182c17172471f6389', 'request_id': 'req-e8b26684-0f90-4890-920f-6dd44302e266'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.727437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca815709d0b4d9182c17172471f6389', 'request_id': 'req-e8b26684-0f90-4890-920f-6dd44302e266'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.734423 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca815709d0b4d9182c17172471f6389', 'request_id': 'req-e8b26684-0f90-4890-920f-6dd44302e266'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.746672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] The token cache is being invalidate because project 2ca815709d0b4d9182c17172471f6389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:32:40.747018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.747701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.749719 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1393/2771] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:32:40 2022] DELETE /identity/v3/projects/2ca815709d0b4d9182c17172471f6389 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:40.754266 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:40.764921 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09027b-8edd-4998-9a8e-5a9c165d2a58 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:40.784576 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:40.824732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:40.825393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:40.825675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:40.825952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] PATH_INFO: `/v3/users/7f49766b24404d7e97864622353df68b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:40.827121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f49766b24404d7e97864622353df68b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:40.829082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:40.875633 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f49766b24404d7e97864622353df68b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.883012 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f49766b24404d7e97864622353df68b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.883770 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f49766b24404d7e97864622353df68b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.884397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f49766b24404d7e97864622353df68b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.891991 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-626e9c36-6c0a-4cdb-82ca-112486a4c000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f49766b24404d7e97864622353df68b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:40.909502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1379/2772] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:32:40 2022] DELETE /identity/v3/users/7f49766b24404d7e97864622353df68b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:41.179272 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.181172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:41.202081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26684-0f90-4890-920f-6dd44302e266 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.272672 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.273439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.273841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.274100 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] PATH_INFO: `/v3/projects/bb29be4c5b3746989e1fda0b71a93d77` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.275449 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb29be4c5b3746989e1fda0b71a93d77)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.277576 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.326722 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb29be4c5b3746989e1fda0b71a93d77', 'request_id': 'req-083ae74a-be6d-40f6-bf22-4428906df242'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:41.341986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb29be4c5b3746989e1fda0b71a93d77', 'request_id': 'req-083ae74a-be6d-40f6-bf22-4428906df242'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:41.349730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb29be4c5b3746989e1fda0b71a93d77', 'request_id': 'req-083ae74a-be6d-40f6-bf22-4428906df242'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:41.360547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] The token cache is being invalidate because project bb29be4c5b3746989e1fda0b71a93d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:32:41.360846 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:41.361515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-083ae74a-be6d-40f6-bf22-4428906df242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:32:41.362985 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1394/2773] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:32:41 2022] DELETE /identity/v3/projects/bb29be4c5b3746989e1fda0b71a93d77 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:41.367810 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.368210 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.368560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.388965 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:41.390252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:41.449300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1380/2774] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:41.458234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.458527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.458803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.478288 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:41.479516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:41.487245 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1395/2775] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:41.491812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.502272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ea7b7-21e3-40c6-898d-10bafd018e0a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.507732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.508325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.508610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.508890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.509542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:32:41.509852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.511472 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.523828 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1381/2776] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:32:41.529763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.539081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf40f42-b7b0-477f-b7fe-b2d56fd3b4b6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.543903 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.544496 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.544779 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.545050 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.545720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.547667 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.574744 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1396/2777] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:41.579186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.589706 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deb3fb-602e-42a0-9e8f-e49ee091618d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.618875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:41.653085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.653694 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.653978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.654252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.654892 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.656817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.678498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1382/2778] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:32:41.682824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.693627 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd81c55-9d14-4ba4-bd36-1e81e8021544 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.698219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.698801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.699076 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.699376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.700041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.701614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.711365 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1397/2779] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:32:41.715312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.725403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014c9a-6cc9-4dcf-b04e-77f9b133db7a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.730103 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.730695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.730980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.731281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.731926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.733512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3483c9-ee34-4609-be50-86cd6e03adfc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.744128 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1383/2780] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:32:41.748170 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.758901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff4663-1339-4733-991d-2542402172b4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.763461 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.764093 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.764379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.764661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] PATH_INFO: `/v3/projects/58275c3cf0cc4377a234efeee5a62e66/users/4655cbed149b40c597d48e7d294ff831/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.778754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58275c3cf0cc4377a234efeee5a62e66, user_id=4655cbed149b40c597d48e7d294ff831, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.781353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:41.783499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:41.794466 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1398/2781] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:41 2022] PUT /identity/v3/projects/58275c3cf0cc4377a234efeee5a62e66/users/4655cbed149b40c597d48e7d294ff831/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:41.805303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.805592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.806000 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.833907 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] MFA Rules not processed for user `4655cbed149b40c597d48e7d294ff831`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:41.835121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:41.904584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1384/2782] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:41.909053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:41.970242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f790f-fa98-46e4-9b51-68dd19be81f7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:41.975447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:41.976041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:41.976322 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:41.976594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:41.977219 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:41.979101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.007511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1399/2783] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:32:41 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:42.011674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.022654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a11f1-b16b-4681-b884-48148389cf43 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.082294 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.082821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.083105 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.083419 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.084062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.085970 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.107696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1385/2784] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:32:42 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:32:42.111941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.122824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d04756-7ebf-4525-ac74-e752289e1695 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.127401 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.127988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.128263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.128555 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.129189 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.131064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.140640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1400/2785] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:32:42.144630 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.154993 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ef47-9e0b-4ec3-9db3-1e361c2c8299 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.159643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.160289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.160571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.161344 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] PATH_INFO: `/v3/projects/5d7a399654494454a5e4d62abf1c0f9c/users/66615c48ab664b56a7965a45bb593d95/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.175801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7a399654494454a5e4d62abf1c0f9c, user_id=66615c48ab664b56a7965a45bb593d95, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.179068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.191538 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1386/2786] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:42 2022] PUT /identity/v3/projects/5d7a399654494454a5e4d62abf1c0f9c/users/66615c48ab664b56a7965a45bb593d95/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:42.196000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.206750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004492-a2d4-49f7-bf71-e63b1f536a9b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.263310 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.263913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.264198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.264473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.265115 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.266710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8930f-5d9d-4806-ba96-61c0333325ee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.276305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1401/2787] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 16:32:42.280593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.287814 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:42.291829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a519ae3-6cb6-4eb7-bf30-0145abd242b0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.297453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.298115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.298392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.298667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] PATH_INFO: `/v3/projects/5d7a399654494454a5e4d62abf1c0f9c/users/66615c48ab664b56a7965a45bb593d95/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.300594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7a399654494454a5e4d62abf1c0f9c, user_id=66615c48ab664b56a7965a45bb593d95, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.303911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.314194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1387/2788] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:32:42 2022] PUT /identity/v3/projects/5d7a399654494454a5e4d62abf1c0f9c/users/66615c48ab664b56a7965a45bb593d95/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:32:42.324457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.324747 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.325043 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.356950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] MFA Rules not processed for user `66615c48ab664b56a7965a45bb593d95`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:32:42.358079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:42.410021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:42.430660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1402/2789] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:32:42.441649 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:42.443458 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:32:42.505306 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f348-dbaf-4f3d-8e59-f8924055286c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:42.509035 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:42.509648 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:42.509925 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:42.510196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:42.566884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:42.568381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:42.577251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1388/2790] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:49.246645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:32:49.257831 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e58e-9da5-4656-8e61-914cec933191 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:32:49.261390 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:32:49.261982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:32:49.262260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:32:49.262531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:32:49.271412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:32:49.272828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11650f09-e03b-42d2-83a4-699f0999cb8c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:32:49.282042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1403/2791] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:32:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:32:52.573194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:32:59.277945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=11 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:19.357346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:19.421032 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cef8a5-7144-4372-8320-3e9e6e92e9e2 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:19.425544 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:19.426165 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:19.426449 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:19.426729 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] PATH_INFO: `/v3/users/4655cbed149b40c597d48e7d294ff831` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:19.428560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4655cbed149b40c597d48e7d294ff831)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:19.430473 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:19.477272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4655cbed149b40c597d48e7d294ff831'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:19.484504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4655cbed149b40c597d48e7d294ff831'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:19.485194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4655cbed149b40c597d48e7d294ff831'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:19.485812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4655cbed149b40c597d48e7d294ff831'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:19.492821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4655cbed149b40c597d48e7d294ff831'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:19.510849 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1389/2792] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:19 2022] DELETE /identity/v3/users/4655cbed149b40c597d48e7d294ff831 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:19.515350 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462224e6-8303-4927-9a95-2954adc8517a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:19.515867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462224e6-8303-4927-9a95-2954adc8517a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:19.515867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462224e6-8303-4927-9a95-2954adc8517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:19.535212 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-462224e6-8303-4927-9a95-2954adc8517a None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:19.536434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-462224e6-8303-4927-9a95-2954adc8517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:19.594804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1404/2793] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:19.605395 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:19.606833 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:19.665835 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f287a4-2631-4b29-bcb4-28791771956b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:19.669802 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:19.670425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:19.670702 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:19.670980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:19.680148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:19.681585 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:19.691106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1390/2794] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:19.962307 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462224e6-8303-4927-9a95-2954adc8517a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:19.963786 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-462224e6-8303-4927-9a95-2954adc8517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:19.979777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462224e6-8303-4927-9a95-2954adc8517a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:19.984994 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:19.985695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:19.985986 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:19.986281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] PATH_INFO: `/v3/projects/58275c3cf0cc4377a234efeee5a62e66` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:19.987541 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=58275c3cf0cc4377a234efeee5a62e66)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:19.989536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.038217 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58275c3cf0cc4377a234efeee5a62e66', 'request_id': 'req-5bacd31a-b78a-42f1-8e1f-5018844053ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.045689 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58275c3cf0cc4377a234efeee5a62e66', 'request_id': 'req-5bacd31a-b78a-42f1-8e1f-5018844053ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.052045 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58275c3cf0cc4377a234efeee5a62e66', 'request_id': 'req-5bacd31a-b78a-42f1-8e1f-5018844053ba'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.062579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] The token cache is being invalidate because project 58275c3cf0cc4377a234efeee5a62e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:20.062872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.063554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.065042 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1405/2795] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:19 2022] DELETE /identity/v3/projects/58275c3cf0cc4377a234efeee5a62e66 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:20.070512 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:20.080200 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058110f-a8da-4781-ad0e-caa0f7eae0cf service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:20.099172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:20.139578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:20.140124 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.140406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.140684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] PATH_INFO: `/v3/users/66615c48ab664b56a7965a45bb593d95` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.141804 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=66615c48ab664b56a7965a45bb593d95)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:20.143744 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.189592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66615c48ab664b56a7965a45bb593d95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.197025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66615c48ab664b56a7965a45bb593d95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.197688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66615c48ab664b56a7965a45bb593d95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.198303 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66615c48ab664b56a7965a45bb593d95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.205067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec001ab-b7ca-4296-a297-aaafb17fe21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66615c48ab664b56a7965a45bb593d95'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.222749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1391/2796] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:20 2022] DELETE /identity/v3/users/66615c48ab664b56a7965a45bb593d95 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:20.503070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:20.505337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:20.523939 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd31a-b78a-42f1-8e1f-5018844053ba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:20.581191 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:20.581817 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.582097 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.582370 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] PATH_INFO: `/v3/projects/5d7a399654494454a5e4d62abf1c0f9c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.583554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7a399654494454a5e4d62abf1c0f9c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:20.585442 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.616779 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:20.634602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7a399654494454a5e4d62abf1c0f9c', 'request_id': 'req-7d6cc649-9101-460f-8064-71172ac124b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.641735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7a399654494454a5e4d62abf1c0f9c', 'request_id': 'req-7d6cc649-9101-460f-8064-71172ac124b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.648222 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7a399654494454a5e4d62abf1c0f9c', 'request_id': 'req-7d6cc649-9101-460f-8064-71172ac124b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.658890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] The token cache is being invalidate because project 5d7a399654494454a5e4d62abf1c0f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:20.659182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.659852 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d6cc649-9101-460f-8064-71172ac124b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:20.661351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1406/2797] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:20 2022] DELETE /identity/v3/projects/5d7a399654494454a5e4d62abf1c0f9c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:20.666891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.667423 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.667564 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.687689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:20.689159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:20.747987 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1392/2798] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:20.756234 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.756515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.756785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.776772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:20.777959 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:20.785742 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1407/2799] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:20.790469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:20.800422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe56154-7ffc-48d1-863a-7baa914d3f34 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:20.806866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:20.807668 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.808079 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.808382 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.809116 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:33:20.809486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:20.811836 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.825312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1393/2800] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:33:20.829885 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:20.839098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1d72e-b229-46f6-85bc-1236f011aa82 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:20.844112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:20.844728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.845011 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.845285 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.845942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:20.847937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.891081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1408/2801] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:20 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:20.895656 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:20.906025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02e82-df6f-4868-a26c-138d3c865de1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:20.932531 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:20.964384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:20.964913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:20.965197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:20.965475 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:20.966111 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:20.968055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:20.989159 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1394/2802] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:20 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:33:20.993316 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.003544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e842232-25a9-4983-8389-c61e6a20b256 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.008447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.009032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.009312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.009611 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.010249 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.011883 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.022240 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1409/2803] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:21.026196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.036213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f54f2-9c6c-43b1-aeb1-cfd21093ec08 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.042119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.042716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.042998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.043314 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.043958 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.045543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f504b2-2667-4128-84d2-73147803d875 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.056304 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1395/2804] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:21.060337 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.070660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e911a7-8da0-41da-b22d-96f55bff3b1a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.075163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.075860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.076139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.076408 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] PATH_INFO: `/v3/projects/a0e3582813ce419e90b0b2cbf3fc1d30/users/55e12fe3df4c40f5be865adf5174d2c9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.093617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e3582813ce419e90b0b2cbf3fc1d30, user_id=55e12fe3df4c40f5be865adf5174d2c9, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.096382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.109969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1410/2805] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:21 2022] PUT /identity/v3/projects/a0e3582813ce419e90b0b2cbf3fc1d30/users/55e12fe3df4c40f5be865adf5174d2c9/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:21.119988 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.120296 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.120599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.148299 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] MFA Rules not processed for user `55e12fe3df4c40f5be865adf5174d2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:21.149524 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:21.218275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1396/2806] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:21.222763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.285184 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbfe20-0331-43e3-985c-03ad8c046dfd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.290411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.291013 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.291330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.291609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.292242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.294143 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.320663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1411/2807] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:21 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:21.324746 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.335927 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b8c2b-49a5-4608-a42d-f05c34ab5b35 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.398477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.399119 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.399444 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.399710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.400346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.402261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.423065 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1397/2808] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:21 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:21.427112 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.437405 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc78f99-e427-409a-9d53-35882912605a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.441890 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.442512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.442789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.443648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.444211 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.445794 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.456094 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1412/2809] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:21.460913 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.470900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd152668-f739-4f2c-b857-838088f2533f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.475922 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.476672 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.476975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.477289 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] PATH_INFO: `/v3/projects/ffdbe1aa68774528971b8bed9e558fab/users/20707b0088804e9d87c5a44a61b50915/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.492422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdbe1aa68774528971b8bed9e558fab, user_id=20707b0088804e9d87c5a44a61b50915, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.495300 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.509977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1398/2810] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:21 2022] PUT /identity/v3/projects/ffdbe1aa68774528971b8bed9e558fab/users/20707b0088804e9d87c5a44a61b50915/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:21.514384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.527460 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d539f-c369-42e4-a169-526f8952305e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.547634 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:21.587074 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.587707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.588018 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.588287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.588922 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.590499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f1207-ac69-4d4b-9d4f-762b0445dd8a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.600182 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1413/2811] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:21.604498 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.609959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:21.616406 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f512c7-3165-4a12-a5fc-bfbfac2910cb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.621442 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.622206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.622489 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.622767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] PATH_INFO: `/v3/projects/ffdbe1aa68774528971b8bed9e558fab/users/20707b0088804e9d87c5a44a61b50915/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.624776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdbe1aa68774528971b8bed9e558fab, user_id=20707b0088804e9d87c5a44a61b50915, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.627470 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.638749 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1399/2812] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:21 2022] PUT /identity/v3/projects/ffdbe1aa68774528971b8bed9e558fab/users/20707b0088804e9d87c5a44a61b50915/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:21.650806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.651089 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.651398 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.679763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] MFA Rules not processed for user `20707b0088804e9d87c5a44a61b50915`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:21.680969 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:21.748613 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1414/2813] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:21.760499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.762274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:21.818945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175db7-996a-4bd2-9391-fbddfa84cd2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.822643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.823244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.823543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.823817 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.832677 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.834132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.843441 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1400/2814] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:21.888485 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:21.889968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:21.902247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902840f-e399-49bc-88ef-f912995e4334 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:21.905920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:21.906511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:21.906796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:21.907065 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:21.963284 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:21.964726 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4347e-f96c-4ce2-90b3-47ad90e63f74 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:21.973945 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1415/2815] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:25.262082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:25.323723 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d9ce-76c2-4de6-ab6c-4e4ea53cddd5 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:25.328284 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:25.328910 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:25.329192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:25.329469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] PATH_INFO: `/v3/users/55e12fe3df4c40f5be865adf5174d2c9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:25.330586 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e12fe3df4c40f5be865adf5174d2c9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:25.332520 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:25.379785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e12fe3df4c40f5be865adf5174d2c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.386617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e12fe3df4c40f5be865adf5174d2c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.387324 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e12fe3df4c40f5be865adf5174d2c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.387951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e12fe3df4c40f5be865adf5174d2c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.395455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e12fe3df4c40f5be865adf5174d2c9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.413455 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1401/2816] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:25 2022] DELETE /identity/v3/users/55e12fe3df4c40f5be865adf5174d2c9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:25.418071 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:25.418582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:25.418582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:25.437645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:25.438867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:25.480279 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:25.498951 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1416/2817] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:25.509293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:25.510716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:25.524862 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:25.570215 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64006b-58f1-40eb-9dd8-1f3ed3d7cf2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:25.573997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:25.574597 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:25.574880 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:25.575154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:25.584752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:25.586171 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:25.595483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1402/2818] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:33:25.864259 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:25.866871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:25.888738 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c5cb-447b-4624-a594-da21a09d3fb5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:25.897645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:25.898277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:25.898479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:25.898881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] PATH_INFO: `/v3/projects/a0e3582813ce419e90b0b2cbf3fc1d30` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:25.899923 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e3582813ce419e90b0b2cbf3fc1d30)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:25.901851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:25.958319 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e3582813ce419e90b0b2cbf3fc1d30', 'request_id': 'req-77fc72c2-cce4-4e87-be61-76c705a04773'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.965604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e3582813ce419e90b0b2cbf3fc1d30', 'request_id': 'req-77fc72c2-cce4-4e87-be61-76c705a04773'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.972152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e3582813ce419e90b0b2cbf3fc1d30', 'request_id': 'req-77fc72c2-cce4-4e87-be61-76c705a04773'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.982832 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] The token cache is being invalidate because project a0e3582813ce419e90b0b2cbf3fc1d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:25.983123 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.983785 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:25.985243 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1417/2819] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:25 2022] DELETE /identity/v3/projects/a0e3582813ce419e90b0b2cbf3fc1d30 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:25.989439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:25.999851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae5c-7456-4094-a017-2f8d7de59f46 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.055028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.055669 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.055944 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.056213 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] PATH_INFO: `/v3/users/20707b0088804e9d87c5a44a61b50915` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.057326 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=20707b0088804e9d87c5a44a61b50915)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.059235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.106517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20707b0088804e9d87c5a44a61b50915'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.113570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20707b0088804e9d87c5a44a61b50915'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.114239 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20707b0088804e9d87c5a44a61b50915'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.115551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20707b0088804e9d87c5a44a61b50915'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.122782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a64d5126-fcb7-4cca-93cb-ffac3af987cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20707b0088804e9d87c5a44a61b50915'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.140569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1403/2820] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:25 2022] DELETE /identity/v3/users/20707b0088804e9d87c5a44a61b50915 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:26.434492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.436031 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:26.449041 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc72c2-cce4-4e87-be61-76c705a04773 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.507409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.508035 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.508314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.508585 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] PATH_INFO: `/v3/projects/ffdbe1aa68774528971b8bed9e558fab` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.509691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdbe1aa68774528971b8bed9e558fab)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.511625 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.557105 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdbe1aa68774528971b8bed9e558fab', 'request_id': 'req-31d01912-6bc7-40d1-8b7d-15650b500112'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.570484 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdbe1aa68774528971b8bed9e558fab', 'request_id': 'req-31d01912-6bc7-40d1-8b7d-15650b500112'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.581608 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdbe1aa68774528971b8bed9e558fab', 'request_id': 'req-31d01912-6bc7-40d1-8b7d-15650b500112'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.591531 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:26.598072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] The token cache is being invalidate because project ffdbe1aa68774528971b8bed9e558fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:26.599135 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.600235 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-31d01912-6bc7-40d1-8b7d-15650b500112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:26.602046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1418/2821] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:26 2022] DELETE /identity/v3/projects/ffdbe1aa68774528971b8bed9e558fab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:26.608051 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.608523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.608643 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.629019 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:26.630967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:26.660432 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:26.691480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1404/2822] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:26.699737 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.700021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.700276 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.720068 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:26.721179 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:26.728868 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1419/2823] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:26.733001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.742620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1918-e0d7-42eb-b4ca-ccb993b391a1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.747151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.747776 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.748062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.748339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.748978 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:33:26.749281 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.751040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.763527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1405/2824] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:33:26.767709 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.777566 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5fbb7-8040-4fc3-a629-a74c5d3ed2a7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.782127 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.782764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.783046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.783351 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.783972 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.785853 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.811975 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1420/2825] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:26 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:26.816192 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.826627 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369de2-7146-4050-a389-cb16090b26f3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.884276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.884887 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.885175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.885454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.886093 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.888056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.919252 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1406/2826] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:26 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:26.923480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.933910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f4d9a-79a9-4039-841f-03b2fe169247 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.939125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.939763 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.940046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.940977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.940977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.942573 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.953330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1421/2827] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:26.957440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:26.968334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9f71d-0828-4cb6-99b2-4d187757d07a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:26.973148 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:26.973680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:26.973966 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:26.974272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:26.974959 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:26.976602 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad99669-53f3-4225-aa66-7e7628aaa906 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:26.987278 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1407/2828] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:26.991811 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.002354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e771610-51eb-4fe6-9ca4-a0a3597eeeee admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.008124 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.008807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.009148 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.009464 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] PATH_INFO: `/v3/projects/20be36c358834448a0547f469e6f81ec/users/151d90d61fb6482d974931a3ff634427/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.024197 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20be36c358834448a0547f469e6f81ec, user_id=151d90d61fb6482d974931a3ff634427, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.026964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.039636 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1422/2829] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:26 2022] PUT /identity/v3/projects/20be36c358834448a0547f469e6f81ec/users/151d90d61fb6482d974931a3ff634427/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:27.049737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29676160-a005-4979-9a34-d038e4820e9d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.050056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29676160-a005-4979-9a34-d038e4820e9d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.050398 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29676160-a005-4979-9a34-d038e4820e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.079331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-29676160-a005-4979-9a34-d038e4820e9d None None] MFA Rules not processed for user `151d90d61fb6482d974931a3ff634427`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:27.080594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-29676160-a005-4979-9a34-d038e4820e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:27.144983 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1408/2830] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:27.149692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.215714 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61e70-b115-40d9-9019-dafd3da5abc7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.220372 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.220977 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.221246 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.221521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.222539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.224604 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.252536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1423/2831] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:27.257508 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29676160-a005-4979-9a34-d038e4820e9d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.268162 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29676160-a005-4979-9a34-d038e4820e9d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.295625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:27.328232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.328840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.329123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.329399 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.330028 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.331974 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.353313 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1409/2832] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:27.357443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.368263 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602b19b-995a-4880-bad5-402f76f551e0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.372804 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.373389 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.373678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.373957 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.374571 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.376177 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.386623 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1424/2833] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:27.390798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.401138 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58554-ccc6-48ea-9cc4-3340ec6b950e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.406003 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.406631 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.406946 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.407248 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.407919 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.409540 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8978b-ef40-445b-91d1-25f03863caa7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.420127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1410/2834] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:27.424297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.434680 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111738b-d985-40ed-a20a-df06dc2df4cd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.439297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.439932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.440210 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.440479 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] PATH_INFO: `/v3/projects/a14171a7bf1e4454bd7f3d8f0fd6aac8/users/ad6409adddcf453791c625abffd04a52/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.454943 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14171a7bf1e4454bd7f3d8f0fd6aac8, user_id=ad6409adddcf453791c625abffd04a52, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.457678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.471277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1425/2835] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:27 2022] PUT /identity/v3/projects/a14171a7bf1e4454bd7f3d8f0fd6aac8/users/ad6409adddcf453791c625abffd04a52/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:27.483276 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.483514 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.483798 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.513070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] MFA Rules not processed for user `ad6409adddcf453791c625abffd04a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:27.514291 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:27.580881 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1411/2836] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:33:27.585517 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.631330 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:27.647859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed7eb-1e2b-4057-bf22-167596e99da2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.652492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.653090 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.653368 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.653638 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.654277 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.656202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.683087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1426/2837] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:27.687272 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.697998 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81297fd7-c62a-4e20-83ea-eb7d2d750760 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.753578 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.754121 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.754657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.754930 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.755583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.757517 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.779163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1412/2838] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:27.783155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.794454 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41f9b-2842-4e68-88ef-bf19c8bf4e55 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.799384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.799988 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.800273 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.800549 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.801190 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.802805 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.813450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1427/2839] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:27.817691 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.827574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dff21-a0b6-46c5-872f-6b23382127cd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.832132 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.832766 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.833045 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.833317 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] PATH_INFO: `/v3/projects/9ae57a2864bd45b6af5f7357e986f288/users/05a9b239a60242de9cbbcb5f65395992/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.848321 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae57a2864bd45b6af5f7357e986f288, user_id=05a9b239a60242de9cbbcb5f65395992, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.851579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.864025 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1413/2840] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:27 2022] PUT /identity/v3/projects/9ae57a2864bd45b6af5f7357e986f288/users/05a9b239a60242de9cbbcb5f65395992/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:27.868019 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.878099 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47456e-e939-4aff-808d-d0494e81eee3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.934999 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.935661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.935942 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.936213 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.936881 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.938463 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0694bae-f2db-49ea-bed0-b67113e67606 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.949651 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1428/2841] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:27.954155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:27.960154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:27.966561 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a08f-de2c-4d8c-b947-aa7bd8242be1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:27.971634 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:27.972309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.972594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.972871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] PATH_INFO: `/v3/projects/9ae57a2864bd45b6af5f7357e986f288/users/05a9b239a60242de9cbbcb5f65395992/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:27.975513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae57a2864bd45b6af5f7357e986f288, user_id=05a9b239a60242de9cbbcb5f65395992, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:27.978130 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:27.988233 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1414/2842] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:27 2022] PUT /identity/v3/projects/9ae57a2864bd45b6af5f7357e986f288/users/05a9b239a60242de9cbbcb5f65395992/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:27.998287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:27.998579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:27.998860 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:28.027209 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] MFA Rules not processed for user `05a9b239a60242de9cbbcb5f65395992`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:28.028373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:28.096157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1429/2843] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:28.106962 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:28.108712 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:28.166717 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02371684-c5ef-4103-a77b-9e8d2d41c0f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:28.170307 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:28.170900 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:28.171177 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:28.171476 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:28.180305 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:28.181713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:28.191168 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1415/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:28.712916 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:28.714354 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:28.726695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba0e3-aafd-4bbd-9477-e1dd5a92f96e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:28.730609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:28.731252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:28.731548 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:28.731822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:28.774784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:28.796404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:28.797955 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:28.808244 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1430/2845] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:32.844911 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:32.853737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a8dd-6705-4a78-98e1-32b9eaf2f8e6 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:32.857562 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:32.858156 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:32.858437 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:32.858710 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:32.917647 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:32.919076 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e79e9-20a7-4963-89c7-59eeac2fdd44 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:32.928338 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1416/2846] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:34.999594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:35.062909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949acaa-2e3b-4a7d-91f8-e12b0a65339c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:35.067808 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:35.068560 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.068872 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.069223 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] PATH_INFO: `/v3/users/151d90d61fb6482d974931a3ff634427` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.070381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] RBAC: Authorizing `identity:delete_user(user_id=151d90d61fb6482d974931a3ff634427)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:35.072347 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:35.119198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151d90d61fb6482d974931a3ff634427'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.126318 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151d90d61fb6482d974931a3ff634427'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.126996 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151d90d61fb6482d974931a3ff634427'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.134493 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151d90d61fb6482d974931a3ff634427'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.146163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151d90d61fb6482d974931a3ff634427'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.152038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1431/2847] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:34 2022] DELETE /identity/v3/users/151d90d61fb6482d974931a3ff634427 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:35.156527 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.157036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.157180 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.176247 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:35.177439 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:35.236840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1417/2848] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:35.246964 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:35.248393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:35.305207 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a7b2-3b69-498f-bf38-164c00ac991b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:35.308789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:35.309383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.309660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.309937 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.319439 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:35.320873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:35.325687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:35.330776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1432/2849] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:35.430902 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:35.432693 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:35.446244 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb673f-8e63-4db0-a816-2e3d65d46ff1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:35.449975 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:35.450573 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.450857 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.451163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.452936 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:35.464964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:35.466855 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:35.479590 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1418/2850] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:35.720910 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:35.722341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:35.735180 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fbb6-fef9-4253-9c31-04d0c69682d9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:35.739818 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:35.740448 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.740730 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.741006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] PATH_INFO: `/v3/projects/20be36c358834448a0547f469e6f81ec` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.742126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=20be36c358834448a0547f469e6f81ec)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:35.744078 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:35.799932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20be36c358834448a0547f469e6f81ec', 'request_id': 'req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.808343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20be36c358834448a0547f469e6f81ec', 'request_id': 'req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.815260 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20be36c358834448a0547f469e6f81ec', 'request_id': 'req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.826579 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] The token cache is being invalidate because project 20be36c358834448a0547f469e6f81ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:35.826870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.827539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.829046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1433/2851] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:35 2022] DELETE /identity/v3/projects/20be36c358834448a0547f469e6f81ec => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:35.833062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:35.844762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428084f-4929-46ab-9782-daf258669518 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:35.857374 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:35.906257 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:35.906891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:35.907172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:35.907502 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] PATH_INFO: `/v3/users/ad6409adddcf453791c625abffd04a52` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:35.908661 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6409adddcf453791c625abffd04a52)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:35.910679 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:35.963655 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6409adddcf453791c625abffd04a52'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.970785 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6409adddcf453791c625abffd04a52'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.971526 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6409adddcf453791c625abffd04a52'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.972163 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6409adddcf453791c625abffd04a52'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.981377 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6409adddcf453791c625abffd04a52'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:35.999801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1419/2852] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:35 2022] DELETE /identity/v3/users/ad6409adddcf453791c625abffd04a52 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:36.290966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:36.292661 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:36.307645 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7738-e0a6-4d0a-bfcb-da42614fa9ce admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:36.371281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:36.371911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:36.372200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:36.372473 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] PATH_INFO: `/v3/projects/a14171a7bf1e4454bd7f3d8f0fd6aac8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:36.373590 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14171a7bf1e4454bd7f3d8f0fd6aac8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:36.375524 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:36.421876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14171a7bf1e4454bd7f3d8f0fd6aac8', 'request_id': 'req-655ab0fc-e2a5-484d-b89b-e5685b739c30'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.429650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14171a7bf1e4454bd7f3d8f0fd6aac8', 'request_id': 'req-655ab0fc-e2a5-484d-b89b-e5685b739c30'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.436066 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14171a7bf1e4454bd7f3d8f0fd6aac8', 'request_id': 'req-655ab0fc-e2a5-484d-b89b-e5685b739c30'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.446224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] The token cache is being invalidate because project a14171a7bf1e4454bd7f3d8f0fd6aac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:36.446513 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.447154 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.448650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1434/2853] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:36 2022] DELETE /identity/v3/projects/a14171a7bf1e4454bd7f3d8f0fd6aac8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:36.452875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:36.454490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:36.467235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed200ff-887a-4454-b615-6641a5868d23 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:36.524155 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:36.524779 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:36.525068 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:36.525346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] PATH_INFO: `/v3/users/05a9b239a60242de9cbbcb5f65395992` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:36.526464 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a9b239a60242de9cbbcb5f65395992)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:36.528407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:36.569194 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:36.577036 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a9b239a60242de9cbbcb5f65395992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.583950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a9b239a60242de9cbbcb5f65395992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.584620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a9b239a60242de9cbbcb5f65395992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.585258 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a9b239a60242de9cbbcb5f65395992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.593908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-52edd0e2-4d75-4d66-8291-25a7baa93271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a9b239a60242de9cbbcb5f65395992'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:36.613129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1420/2854] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:36 2022] DELETE /identity/v3/users/05a9b239a60242de9cbbcb5f65395992 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:36.904702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:36.906338 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:36.922803 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ab0fc-e2a5-484d-b89b-e5685b739c30 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:36.933104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:36.998289 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:36.998920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:36.999201 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:36.999504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] PATH_INFO: `/v3/projects/9ae57a2864bd45b6af5f7357e986f288` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.000633 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae57a2864bd45b6af5f7357e986f288)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.002544 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.048305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae57a2864bd45b6af5f7357e986f288', 'request_id': 'req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:37.056287 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae57a2864bd45b6af5f7357e986f288', 'request_id': 'req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:37.062139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae57a2864bd45b6af5f7357e986f288', 'request_id': 'req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:37.073404 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] The token cache is being invalidate because project 9ae57a2864bd45b6af5f7357e986f288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:37.073572 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:37.074140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-67b1ecf7-cdfd-4cb6-b4c9-9edb031c0e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:37.075632 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1435/2855] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:36 2022] DELETE /identity/v3/projects/9ae57a2864bd45b6af5f7357e986f288 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:37.081670 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.081960 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.082236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.103599 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:37.104860 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:37.164362 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1421/2856] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:37.174061 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.174355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.174635 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.195046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:37.196304 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:37.204411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1436/2857] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:37.208840 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.219136 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4065e7-3eef-4886-9db3-7023241ef3ed None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.223753 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.224349 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.225114 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.225393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.226056 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:33:37.226358 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.227981 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.239595 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1422/2858] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:33:37.244515 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.253843 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465929-71ad-48c1-b246-575940c9b6ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.258707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.259336 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.259620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.259898 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.260553 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.262447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.289410 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1437/2859] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:37.293588 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.304264 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d057-5f4d-4ea1-9a71-3dda76ae3850 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.359784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.360309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.360593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.360869 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.361504 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.363452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.386023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1423/2860] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:33:37.390181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.400432 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f488b-3965-451b-a8f9-c8b35c0957f5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.405145 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.405740 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.406021 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.406297 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.406944 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.408878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.419296 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1438/2861] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:37.423430 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.433858 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f690-39d2-44bd-9430-efdbf5a837a1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.438499 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.439092 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.439407 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.439683 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.440322 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.441909 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715554b-1320-4fa0-b4f7-432e6e4b84ec admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.452519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1424/2862] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:37.456641 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.467139 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c4aa7-8fd6-403c-a474-7a26bd2e6248 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.471777 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.472416 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.472701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.472978 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] PATH_INFO: `/v3/projects/bc288d32dfbd48209de22a5b3e48dd98/users/356d0036a63d4c059cb50a553af8927f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.488367 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc288d32dfbd48209de22a5b3e48dd98, user_id=356d0036a63d4c059cb50a553af8927f, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.491046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.504006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1439/2863] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:37 2022] PUT /identity/v3/projects/bc288d32dfbd48209de22a5b3e48dd98/users/356d0036a63d4c059cb50a553af8927f/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:37.515805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.516334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.516457 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.545363 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] MFA Rules not processed for user `356d0036a63d4c059cb50a553af8927f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:37.546575 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:37.556031 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:37.614021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1425/2864] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:37.618456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.662997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:37.684355 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6382bc-7405-4cb9-a061-85ee759cde72 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.689046 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.689801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.690087 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.690455 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.691004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.692974 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.722051 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1440/2865] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:37.726806 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.737477 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385762f4-0eb2-45f4-8704-d0ca4a2d60c1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.793876 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.794486 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.794768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.795048 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.795731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.797660 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.819821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1426/2866] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:37 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:37.823834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.833906 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b49f92-3c7f-48a8-b182-737f9d528b11 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.839001 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.839620 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.839901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.840172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.840806 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.842387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.852687 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1441/2867] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:37.856642 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.866713 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c904fb-3bac-4e33-9656-266cf9dd87f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.871255 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.871908 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.872196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.872480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] PATH_INFO: `/v3/projects/77b6e131409444c997f35271fff445c6/users/4e81414771964c06907b0015cb1dd4b2/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.888950 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b6e131409444c997f35271fff445c6, user_id=4e81414771964c06907b0015cb1dd4b2, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.891644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.904884 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1427/2868] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:37 2022] PUT /identity/v3/projects/77b6e131409444c997f35271fff445c6/users/4e81414771964c06907b0015cb1dd4b2/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:37.908896 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:37.918927 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c95df-7b38-4059-98db-bb587e47f67c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:37.975492 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:37.976098 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:37.976378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:37.976650 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:37.977293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:37.978889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149e41f-e889-45f4-a60f-6d0b2bf489b7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:37.990302 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1442/2869] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:37.994768 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:38.005415 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8fbc-29ae-44ca-a032-6069d8c9dabb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:38.010584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:38.011249 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:38.011549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:38.011826 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] PATH_INFO: `/v3/projects/77b6e131409444c997f35271fff445c6/users/4e81414771964c06907b0015cb1dd4b2/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:38.013761 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b6e131409444c997f35271fff445c6, user_id=4e81414771964c06907b0015cb1dd4b2, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:38.016434 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:38.028259 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1428/2870] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:37 2022] PUT /identity/v3/projects/77b6e131409444c997f35271fff445c6/users/4e81414771964c06907b0015cb1dd4b2/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:38.038343 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:38.038637 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:38.038952 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:38.066039 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] MFA Rules not processed for user `4e81414771964c06907b0015cb1dd4b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:38.067282 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:38.136751 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1443/2871] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:38.149469 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:38.151384 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:38.209381 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964a0e7-eb4b-40a8-af7b-3bec462205b7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:38.213569 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:38.214175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:38.214456 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:38.214730 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:38.231410 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:38.272587 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:38.274013 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d2434-a8fe-4176-8079-54400853e452 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:38.282842 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1429/2872] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:40.822562 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:40.869256 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:40.894158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9268ad-b18c-4318-8f26-af3250dd6b17 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:40.898718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:40.899313 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:40.899599 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:40.899876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] PATH_INFO: `/v3/users/356d0036a63d4c059cb50a553af8927f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:40.900998 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=356d0036a63d4c059cb50a553af8927f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:40.902932 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:40.953440 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356d0036a63d4c059cb50a553af8927f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:40.960373 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356d0036a63d4c059cb50a553af8927f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:40.961036 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356d0036a63d4c059cb50a553af8927f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:40.968592 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356d0036a63d4c059cb50a553af8927f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:40.981476 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356d0036a63d4c059cb50a553af8927f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:40.987314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1444/2873] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:40 2022] DELETE /identity/v3/users/356d0036a63d4c059cb50a553af8927f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:40.991931 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:40.993049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:40.993164 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:41.012235 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:41.013536 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:41.073696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1430/2874] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:41.083403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:41.084807 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:41.141669 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ae46-626e-42fa-a992-487b545fe195 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:41.145320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:41.145921 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:41.146205 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:41.146480 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:41.155920 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:41.157290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:41.167281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1445/2875] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:41.442072 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:41.443539 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:41.456188 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e196f2-d441-4aa2-8c71-3c1437930df8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:41.460740 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:41.461859 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:41.462137 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:41.462411 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] PATH_INFO: `/v3/projects/bc288d32dfbd48209de22a5b3e48dd98` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:41.463563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc288d32dfbd48209de22a5b3e48dd98)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:41.465914 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:41.514143 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc288d32dfbd48209de22a5b3e48dd98', 'request_id': 'req-e749c017-9ef4-40ee-a49d-c630dd575975'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.520490 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc288d32dfbd48209de22a5b3e48dd98', 'request_id': 'req-e749c017-9ef4-40ee-a49d-c630dd575975'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.527038 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc288d32dfbd48209de22a5b3e48dd98', 'request_id': 'req-e749c017-9ef4-40ee-a49d-c630dd575975'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.540049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] The token cache is being invalidate because project bc288d32dfbd48209de22a5b3e48dd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:41.540339 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.540977 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.542466 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1431/2876] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:41 2022] DELETE /identity/v3/projects/bc288d32dfbd48209de22a5b3e48dd98 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:41.546664 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:41.556152 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8808ca-f187-46ca-8bc6-ce14d2aedae5 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:41.606882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:41.613798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:41.614444 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:41.615058 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:41.615443 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] PATH_INFO: `/v3/users/4e81414771964c06907b0015cb1dd4b2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:41.616793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e81414771964c06907b0015cb1dd4b2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:41.619293 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:41.672847 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e81414771964c06907b0015cb1dd4b2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.683642 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e81414771964c06907b0015cb1dd4b2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.684406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e81414771964c06907b0015cb1dd4b2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.693204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e81414771964c06907b0015cb1dd4b2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.710931 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-57af4839-4168-44ac-b3cd-91ee8d5b1e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e81414771964c06907b0015cb1dd4b2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:41.718192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1446/2877] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:41 2022] DELETE /identity/v3/users/4e81414771964c06907b0015cb1dd4b2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:41.987418 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:41.988868 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:42.002834 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c017-9ef4-40ee-a49d-c630dd575975 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.045543 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:42.065752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.066392 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.066676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.066951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] PATH_INFO: `/v3/projects/77b6e131409444c997f35271fff445c6` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.068172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b6e131409444c997f35271fff445c6)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.070274 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.116106 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b6e131409444c997f35271fff445c6', 'request_id': 'req-6632a669-aba0-4a9d-aea4-29b10b06f0a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:42.122797 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b6e131409444c997f35271fff445c6', 'request_id': 'req-6632a669-aba0-4a9d-aea4-29b10b06f0a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:42.128308 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b6e131409444c997f35271fff445c6', 'request_id': 'req-6632a669-aba0-4a9d-aea4-29b10b06f0a2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:42.141320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] The token cache is being invalidate because project 77b6e131409444c997f35271fff445c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:42.141606 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:42.142232 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6632a669-aba0-4a9d-aea4-29b10b06f0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:42.143738 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1432/2878] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:41 2022] DELETE /identity/v3/projects/77b6e131409444c997f35271fff445c6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:42.149961 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.150446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.150584 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.169947 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:42.171140 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:42.229017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1447/2879] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:42.236718 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.237017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.237355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.258756 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:42.259990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:42.267845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1433/2880] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:42.272069 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.281741 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5161-ac9d-4037-a59f-42755d9f7b7f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.286281 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.286865 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.287142 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.287451 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.288092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:33:42.288394 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.290873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.302696 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1448/2881] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 16:33:42.308403 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.320971 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf66c1-ec55-4841-9130-75740c0c68d2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.329214 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.329821 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.330102 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.330570 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.331681 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.334820 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.373589 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1434/2882] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:42.377954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.388429 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaafacc-08dd-4d5c-9ca9-772698010d93 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.444363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.444968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.445252 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.445698 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.446339 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.448376 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.458206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:42.470997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1449/2883] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:42.475085 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.485895 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078163-5fee-49f5-90eb-29cb12755bb4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.490574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.491113 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.491431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.491708 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.492947 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.494551 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.504915 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1435/2884] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:42.508793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.518987 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899fad3-158f-4951-a9e1-aa44ef5f171e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.523660 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.524250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.524528 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.524798 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.525438 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.527038 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafc370-100c-4cc2-97aa-0ce4b9262cc3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.537602 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1450/2885] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:42.542228 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.552206 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513ed6-346f-40f2-b8be-81198272de2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.557453 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.558101 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.558387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.558664 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] PATH_INFO: `/v3/projects/25def820fbd144fa8fbf3dc3252affe9/users/a943be4e410043fd90046b5492ef4559/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.573127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25def820fbd144fa8fbf3dc3252affe9, user_id=a943be4e410043fd90046b5492ef4559, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.575818 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.588366 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1436/2886] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:42 2022] PUT /identity/v3/projects/25def820fbd144fa8fbf3dc3252affe9/users/a943be4e410043fd90046b5492ef4559/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:42.599126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.599477 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.599750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.629767 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] MFA Rules not processed for user `a943be4e410043fd90046b5492ef4559`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:42.631017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:42.699617 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1451/2887] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:42.704186 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.765732 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9560-1a4f-4b76-94a0-15417e04710e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.770654 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.771285 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.771574 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.771852 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.772488 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.774891 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.789141 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:42.802943 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1437/2888] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:42.807082 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.818266 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f77781-0b1a-43a9-9252-19d5b66f010c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.874980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.875542 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.875828 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.876104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.876750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.878691 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.900299 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1452/2889] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:42 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:42.904408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.915704 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997d48a-dfd1-4016-a347-e1cd59232b06 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.920640 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.921229 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.921519 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.921796 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.922440 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.924062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.934724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1438/2890] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:42.938784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.950729 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854657-d188-4c64-81b3-d5b28ad67a97 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.956169 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.956772 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.957064 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.957331 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:42.957976 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:42.959609 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db283e-0aad-4f2d-9192-fec7a1eafeb9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:42.970624 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1453/2891] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:42.974750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:42.985202 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f1d30-4f69-4b52-b046-c17040e73524 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:42.989763 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:42.990400 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:42.990676 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:42.990948 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] PATH_INFO: `/v3/projects/411d03036ce243c8b1424043cd3d8d07/users/e54f5a79192d4163848c2a11754adab2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.005622 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=411d03036ce243c8b1424043cd3d8d07, user_id=e54f5a79192d4163848c2a11754adab2, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.008790 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.021967 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1439/2892] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:42 2022] PUT /identity/v3/projects/411d03036ce243c8b1424043cd3d8d07/users/e54f5a79192d4163848c2a11754adab2/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:43.033291 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.033612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.033889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.061384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] MFA Rules not processed for user `e54f5a79192d4163848c2a11754adab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:43.062586 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:43.103830 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:43.129022 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1454/2893] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:43.134480 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.196805 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460aab7-1d23-4d50-a45b-3de518079fab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.201513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.202112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.202397 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.202675 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.203346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.205240 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.233133 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1440/2894] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:43 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 16:33:43.236784 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.247356 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141a6f0-e0e1-4634-8dd4-225d13d31657 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.304718 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.305251 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.305532 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.305800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.306418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.308399 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.332715 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1455/2895] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:43 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:43.333945 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.345267 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d01978-357d-4b42-8219-87b663c042b3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.349972 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.350565 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.350850 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.351127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.351813 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.353402 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.364199 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1441/2896] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:43.368889 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.378681 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a193e1-949e-4ec3-bc58-03514773cbef admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.384059 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.384701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.384980 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.385253 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] PATH_INFO: `/v3/projects/b17d162de18d4859b9260c694a12a0bc/users/c91760794b27435cb5302648f5ff159b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.400436 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17d162de18d4859b9260c694a12a0bc, user_id=c91760794b27435cb5302648f5ff159b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.403070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.416028 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1456/2897] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:43 2022] PUT /identity/v3/projects/b17d162de18d4859b9260c694a12a0bc/users/c91760794b27435cb5302648f5ff159b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:43.419949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.429711 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306343c0-0c19-4ccf-baee-a0080a0d63c5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.485461 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.486058 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.486336 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.486607 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.487261 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.488829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284cd6a-7bf4-46db-bb87-395a9b218c83 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.499061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1442/2898] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:43.503344 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.514554 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6296e-d3e3-4423-9b4c-6f9dfe031447 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.519172 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.519849 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.520155 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.520428 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] PATH_INFO: `/v3/projects/b17d162de18d4859b9260c694a12a0bc/users/c91760794b27435cb5302648f5ff159b/roles/8375c8e5926547ccbdfc1e86457e975a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.522312 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17d162de18d4859b9260c694a12a0bc, user_id=c91760794b27435cb5302648f5ff159b, role_id=8375c8e5926547ccbdfc1e86457e975a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.525032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.535300 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1457/2899] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:43 2022] PUT /identity/v3/projects/b17d162de18d4859b9260c694a12a0bc/users/c91760794b27435cb5302648f5ff159b/roles/8375c8e5926547ccbdfc1e86457e975a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:43.545275 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.545592 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.545871 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.576364 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] MFA Rules not processed for user `c91760794b27435cb5302648f5ff159b`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:43.577513 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:43.593980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:43.652688 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1443/2900] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:43.663873 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:43.665764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:43.725153 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95349c45-e2f1-4c2b-bee8-d42be24e7bfd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:43.728800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:43.729393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:43.729670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:43.729941 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:43.773867 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:43.785886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:43.787363 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:43.796580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1458/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:45.302632 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:45.304066 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:45.316501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07dc09-4e7b-46bd-ad2a-53224fe1ff3c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:45.320030 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:45.320620 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:45.320898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:45.321169 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:45.375980 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:45.377312 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:45.386532 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1444/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:45.924692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:45.934030 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb597bac-809c-4872-a9dd-48ce2024b19f service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:45.937673 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:45.938268 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:45.938547 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:45.938822 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:45.947851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:45.949274 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8845b5d-bd78-4a4d-8d50-fe41a0331f49 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:45.958901 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1459/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:49.817445 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:49.880223 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ed84-cc9d-4c2a-91a2-4255baf12303 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:49.884819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:49.885452 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:49.885737 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:49.886015 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] PATH_INFO: `/v3/users/a943be4e410043fd90046b5492ef4559` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:49.887144 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a943be4e410043fd90046b5492ef4559)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:49.889099 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:49.935550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a943be4e410043fd90046b5492ef4559'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:49.940951 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:49.943191 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a943be4e410043fd90046b5492ef4559'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:49.944197 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a943be4e410043fd90046b5492ef4559'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:49.945282 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a943be4e410043fd90046b5492ef4559'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:49.952973 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a943be4e410043fd90046b5492ef4559'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:49.969964 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1445/2904] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:49 2022] DELETE /identity/v3/users/a943be4e410043fd90046b5492ef4559 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:49.974383 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:49.974884 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:49.974993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:49.994938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:49.996156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:50.053025 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1460/2905] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:50.063523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:50.064985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:50.124355 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174ad8-4bec-449d-bf31-0048d7d1d9aa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:50.128172 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:50.128773 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:50.129055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:50.129593 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:50.138741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:50.140196 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:50.150081 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1446/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:50.419603 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:50.421167 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:50.435003 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fce3c8-ac78-4e17-8b5b-43be46b12909 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:50.440360 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:50.441081 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:50.441278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:50.441582 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] PATH_INFO: `/v3/projects/25def820fbd144fa8fbf3dc3252affe9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:50.442710 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=25def820fbd144fa8fbf3dc3252affe9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:50.444600 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:50.491168 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25def820fbd144fa8fbf3dc3252affe9', 'request_id': 'req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.498450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25def820fbd144fa8fbf3dc3252affe9', 'request_id': 'req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.505335 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25def820fbd144fa8fbf3dc3252affe9', 'request_id': 'req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.516192 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] The token cache is being invalidate because project 25def820fbd144fa8fbf3dc3252affe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:50.516499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.517136 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.518619 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1461/2907] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:50 2022] DELETE /identity/v3/projects/25def820fbd144fa8fbf3dc3252affe9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:50.522667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:50.533123 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5d6ac-abb3-4fd9-8dfe-f3c4bcfcf2f7 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:50.590049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:50.590680 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:50.590963 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:50.591260 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] PATH_INFO: `/v3/users/e54f5a79192d4163848c2a11754adab2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:50.592387 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e54f5a79192d4163848c2a11754adab2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:50.594334 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:50.640064 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e54f5a79192d4163848c2a11754adab2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.651017 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e54f5a79192d4163848c2a11754adab2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.651743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e54f5a79192d4163848c2a11754adab2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.652348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e54f5a79192d4163848c2a11754adab2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.660023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e54f5a79192d4163848c2a11754adab2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:50.670047 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:50.678743 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1447/2908] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:50 2022] DELETE /identity/v3/users/e54f5a79192d4163848c2a11754adab2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 16:33:50.959512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:50.961000 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:50.973731 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925e39-5d46-4ee2-8f58-068ce8a6c2aa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:51.028834 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:51.035736 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:51.036434 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.036981 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.037426 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] PATH_INFO: `/v3/projects/411d03036ce243c8b1424043cd3d8d07` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.038764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=411d03036ce243c8b1424043cd3d8d07)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:51.040814 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:51.087851 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411d03036ce243c8b1424043cd3d8d07', 'request_id': 'req-6306c7c0-2808-48f9-924c-9a730d3f0c56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.095006 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411d03036ce243c8b1424043cd3d8d07', 'request_id': 'req-6306c7c0-2808-48f9-924c-9a730d3f0c56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.101183 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411d03036ce243c8b1424043cd3d8d07', 'request_id': 'req-6306c7c0-2808-48f9-924c-9a730d3f0c56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.111403 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] The token cache is being invalidate because project 411d03036ce243c8b1424043cd3d8d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:51.111692 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.112320 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.113778 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1462/2909] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:50 2022] DELETE /identity/v3/projects/411d03036ce243c8b1424043cd3d8d07 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:51.118353 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:51.119875 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:51.133872 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95b9a-ac2e-49fb-b0ac-61b0fb75ee29 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:51.189503 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:51.190129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.190414 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.190689 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] PATH_INFO: `/v3/users/c91760794b27435cb5302648f5ff159b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.191829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91760794b27435cb5302648f5ff159b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:51.193741 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:51.240127 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91760794b27435cb5302648f5ff159b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.247560 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91760794b27435cb5302648f5ff159b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.248236 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91760794b27435cb5302648f5ff159b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.248851 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91760794b27435cb5302648f5ff159b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.261716 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f29d0732-8909-4a58-824b-ba50d16785ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91760794b27435cb5302648f5ff159b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.281293 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1448/2910] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:33:51 2022] DELETE /identity/v3/users/c91760794b27435cb5302648f5ff159b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:51.563421 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:51.564855 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:51.578735 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306c7c0-2808-48f9-924c-9a730d3f0c56 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:51.637909 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:51.638578 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.638859 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.639382 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] PATH_INFO: `/v3/projects/b17d162de18d4859b9260c694a12a0bc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.640629 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17d162de18d4859b9260c694a12a0bc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:51.642707 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:51.690385 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17d162de18d4859b9260c694a12a0bc', 'request_id': 'req-b1742314-7b6c-4d4c-a573-47bd0e929ac0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.697386 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17d162de18d4859b9260c694a12a0bc', 'request_id': 'req-b1742314-7b6c-4d4c-a573-47bd0e929ac0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.703678 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17d162de18d4859b9260c694a12a0bc', 'request_id': 'req-b1742314-7b6c-4d4c-a573-47bd0e929ac0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.714504 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] The token cache is being invalidate because project b17d162de18d4859b9260c694a12a0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:33:51.714792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.715452 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b1742314-7b6c-4d4c-a573-47bd0e929ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:33:51.716911 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1463/2911] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:33:51 2022] DELETE /identity/v3/projects/b17d162de18d4859b9260c694a12a0bc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:51.722040 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.722333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.722610 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.742617 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:51.743844 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:51.804376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1449/2912] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 16:33:51.812498 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.812764 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.812973 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.835054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:51.835054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:51.843607 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1464/2913] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:51.847893 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:51.858346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a0d3d-8fb1-42d2-8d9d-ce55ddd1a13c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:51.863043 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:51.863663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.863949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.864224 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.864874 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 16:33:51.865174 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:51.866769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:51.875762 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:51.883638 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1450/2914] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 30 16:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 16:33:51.884750 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:51.894054 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33588bbc-c6d9-44d2-803a-4d618b4da428 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:51.899278 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:51.899875 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:51.900157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:51.900430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:51.901070 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:51.902954 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:51.912447 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:33:51.930891 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1465/2915] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 30 16:33:51 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:51.935352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:51.945674 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead1369-79cd-48e8-b215-663505bbd1b5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:52.001501 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:52.002112 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.002385 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.002687 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.003421 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:52.005616 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:52.026772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1451/2916] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 30 16:33:51 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 16:33:52.030894 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:52.041242 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c03c80-8944-4ec7-acc0-bd9ffacb22fb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:52.045720 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:52.046305 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.046580 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.046850 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.047527 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:52.049109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:52.058886 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1466/2917] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:52.062791 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:52.073297 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a70930-78f1-4434-8035-c76470d21098 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:52.078005 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:52.078594 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.078898 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.079170 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.079845 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:52.081422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f994c40-72de-4aae-8261-79388201bbe1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:52.092492 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1452/2918] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 30 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 16:33:52.096499 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:52.106536 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737b94-e0d5-4941-ad41-dda502b80752 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:52.111158 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:52.111848 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.112125 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.112397 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] PATH_INFO: `/v3/projects/abfbc1917a984eca86d589d76c39faa8/users/db5e739e53ea446481597fa1e718b434/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.126892 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfbc1917a984eca86d589d76c39faa8, user_id=db5e739e53ea446481597fa1e718b434, role_id=2b3f90e5c1c74f67983c0ec9b8f9ab31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:52.129800 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:52.144384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1467/2919] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 30 16:33:52 2022] PUT /identity/v3/projects/abfbc1917a984eca86d589d76c39faa8/users/db5e739e53ea446481597fa1e718b434/roles/2b3f90e5c1c74f67983c0ec9b8f9ab31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:33:52.150351 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2a8f-cfa4-4ad9-9213-c17acfe6c7fe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.150916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2a8f-cfa4-4ad9-9213-c17acfe6c7fe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.150916 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a2a8f-cfa4-4ad9-9213-c17acfe6c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.172417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8a9a2a8f-cfa4-4ad9-9213-c17acfe6c7fe None None] MFA Rules not processed for user `b24720c8535f4d608668a1e3b4a4419c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:52.173636 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8a9a2a8f-cfa4-4ad9-9213-c17acfe6c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:52.233001 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1453/2920] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:52.242662 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:52.244419 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:52.301465 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c08f6-d450-4404-aa7a-8322e4e04411 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:52.305379 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:52.306359 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.306870 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.307411 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.318200 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:52.319614 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:52.329574 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1468/2921] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:52.953641 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f41c6-b218-4393-b324-c8363e9d62f5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:52.953935 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f41c6-b218-4393-b324-c8363e9d62f5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:52.954217 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f41c6-b218-4393-b324-c8363e9d62f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:52.982523 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f28f41c6-b218-4393-b324-c8363e9d62f5 None None] MFA Rules not processed for user `db5e739e53ea446481597fa1e718b434`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:52.983949 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f28f41c6-b218-4393-b324-c8363e9d62f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:53.053750 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1454/2922] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:33:53.065101 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:33:53.066539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:53.127728 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708748ed-4e11-4411-909a-21702e9df5cf service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:33:53.131563 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:33:53.132163 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:53.132450 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:53.132758 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:53.141236 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:33:53.142670 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:33:53.152121 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1469/2923] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:33:54.505082 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc65a-797d-438e-90bf-8e35835a528c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:33:54.505703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc65a-797d-438e-90bf-8e35835a528c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:33:54.505703 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc65a-797d-438e-90bf-8e35835a528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:33:54.526550 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-68afc65a-797d-438e-90bf-8e35835a528c None None] MFA Rules not processed for user `db5e739e53ea446481597fa1e718b434`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:33:54.528393 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-68afc65a-797d-438e-90bf-8e35835a528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:33:54.540724 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1455/2924] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:34:01.409418 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:01.411196 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:01.474358 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:01.498773 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d9565-be14-4ca4-bc04-5bfeadb7dea6 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:01.505799 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:01.506412 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:01.506695 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:01.506966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:01.509353 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:01.522221 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:01.523815 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:01.535110 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1470/2925] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:34:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:34:04.520801 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:11.531516 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:15.436990 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:15.437546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:15.437546 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:15.456348 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] MFA Rules not processed for user `db5e739e53ea446481597fa1e718b434`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:34:15.457549 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:15.464426 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1456/2926] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:34:15.702092 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:15.703539 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:15.715640 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d311a-2eba-4513-8f08-ab30e11343b3 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:15.719198 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:15.719819 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:15.720103 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:15.720378 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:15.722063 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:15.732032 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:15.733457 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:15.741409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1471/2927] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:34:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:34:16.087253 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:16.088684 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:16.165149 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cbab8-3cab-4d79-bf92-08b753355622 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:16.170145 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:16.170752 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:16.171039 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:16.171352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:16.173061 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:16.184471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:16.185985 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852da250-6bc4-4e02-9fc8-8c9a908afb7c service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:16.196450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1457/2928] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:34:25.055017 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:25.069514 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528ac8-43d9-46fa-a64d-828a78db55d3 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:25.075882 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:25.076495 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:25.076792 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:25.077053 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:25.078810 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:25.091801 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:25.093409 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:25.105250 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1472/2929] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:34:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:34:26.191667 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=35 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:35.099946 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=12 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:37.599042 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:37.599612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:37.599612 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:37.657273 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:34:37.658493 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:37.726175 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1458/2930] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 30 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:34:37.777437 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:37.778950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:37.877391 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c6c5b-08cb-447f-8af3-1942e3eef36c service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:37.882876 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:37.883511 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:37.883797 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:37.884072 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:37.892456 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:37.894048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b7ec-db70-4f8e-abe3-266f86022d2c service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:37.902231 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1473/2931] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:34:46.315657 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:46.317190 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:46.329542 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dd4ba-8277-4095-94a5-86b640e5185a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:46.333450 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de5581201518429191f74cf8af19fb58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:46.334408 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:46.334696 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:46.335663 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:46.337371 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:46.406433 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:46.408055 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:46.436396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1459/2932] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:34:47.899157 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=41 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:55.353825 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c29-8a39-43d4-9638-d77b6b5860bc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:55.354387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c29-8a39-43d4-9638-d77b6b5860bc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:55.354387 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c29-8a39-43d4-9638-d77b6b5860bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:55.373126 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7e681c29-8a39-43d4-9638-d77b6b5860bc None None] MFA Rules not processed for user `53087055f7e0417b99552e16781f4542`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:34:55.374314 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e681c29-8a39-43d4-9638-d77b6b5860bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:55.381598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1474/2933] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 30 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:34:55.393563 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:55.405725 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f796e0-d31c-4401-bfb1-77ca59c0c43c service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:55.411125 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '058f62fc005a41b08ee30fb42192e31a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:55.411765 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:55.412049 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:55.412325 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:55.427772 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:55.429352 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:55.437812 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1460/2934] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 30 16:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 16:34:57.621446 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:57.622004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:57.622004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:57.642630 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] MFA Rules not processed for user `20cf8769967d4169be2a2a82431c7df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 16:34:57.643878 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:34:57.651594 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1475/2935] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 30 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 16:34:58.454330 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:58.467721 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d76e9d-b759-431a-83f3-f26f6254331e service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:58.475396 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:58.476024 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:58.476333 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:58.476572 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] PATH_INFO: `/v3/users/db5e739e53ea446481597fa1e718b434` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:58.477890 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5e739e53ea446481597fa1e718b434)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:58.480754 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:58.539067 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5e739e53ea446481597fa1e718b434'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.546023 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:34:58.552021 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5e739e53ea446481597fa1e718b434'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.552792 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5e739e53ea446481597fa1e718b434'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.553926 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5e739e53ea446481597fa1e718b434'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.564623 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5e739e53ea446481597fa1e718b434'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.587644 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1461/2936] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:34:58 2022] DELETE /identity/v3/users/db5e739e53ea446481597fa1e718b434 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:34:58.783991 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:34:58.798392 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cd6e3-7a04-454b-99a9-62bb15f64096 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:34:58.876754 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:34:58.877393 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:34:58.877668 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:34:58.877938 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] PATH_INFO: `/v3/projects/abfbc1917a984eca86d589d76c39faa8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:34:58.879079 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=abfbc1917a984eca86d589d76c39faa8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:34:58.881327 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:34:58.935301 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abfbc1917a984eca86d589d76c39faa8', 'request_id': 'req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.945701 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abfbc1917a984eca86d589d76c39faa8', 'request_id': 'req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.952950 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abfbc1917a984eca86d589d76c39faa8', 'request_id': 'req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.966481 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] The token cache is being invalidate because project abfbc1917a984eca86d589d76c39faa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:34:58.966776 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.967512 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:34:58.968997 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1476/2937] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:34:58 2022] DELETE /identity/v3/projects/abfbc1917a984eca86d589d76c39faa8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:35:08.583583 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=10,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:35:08.963024 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:35:16.445920 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:35:16.447422 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:35:16.506115 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409c11f-6ec3-46ff-bfae-189b0c1ea3cf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:35:16.554547 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:35:16.555150 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:35:16.555454 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:35:16.555731 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:35:16.557431 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:35:16.619767 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:35:16.621376 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:35:16.632070 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1462/2938] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:35:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:35:26.626666 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=70 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:36:32.701841 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:36:32.703247 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:36:32.841829 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e32bd-0c0d-4783-b2fe-e3f4d62902cc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:36:32.847020 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:36:32.847648 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:36:32.847930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:36:32.848204 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:36:32.849935 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:36:32.861396 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:36:32.862968 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:36:32.874793 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1477/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:36:42.868760 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=63 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:37:08.475984 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:08.478332 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:08.686331 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a09de-13d1-4288-90f0-721a00693b8d service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:08.693309 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce87b2207504e839bcd544faa3aa7ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:08.694365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:08.694365 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:08.694604 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:08.696280 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:08.710185 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:08.711889 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:08.723695 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1463/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:37:16.846663 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:16.848104 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:16.887824 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76edd98-5336-48e8-ae17-512251ab7af0 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:16.891966 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:16.892559 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:16.892838 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:16.893109 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:16.894809 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:16.908494 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:16.909913 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:16.920570 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1478/2941] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:37:18.719375 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=68 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:37:21.089626 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:21.099819 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227fc6-cc6e-4f8c-87d8-530fb0e0e95b service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:21.103468 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393f32c7fcdf49219d155bb22fd3d048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:21.104062 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:21.104346 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:21.104625 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:21.106357 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:21.192782 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:21.194129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:21.213004 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1464/2942] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:37:24.012341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:24.013930 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:24.030083 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd77b5-a77f-4a65-a57d-43188cc9e57b service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:24.035675 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:24.036224 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:24.036796 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:24.037248 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:24.041181 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:24.061598 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:24.066004 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:24.086341 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1479/2943] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:37:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 16:37:27.328075 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:27.329614 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:27.343404 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45107ec1-3532-4e3f-994f-be67c483ac2c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:27.348212 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:27.348870 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:27.349151 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:27.349425 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] PATH_INFO: `/v3/users/20cf8769967d4169be2a2a82431c7df6` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:27.350548 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=20cf8769967d4169be2a2a82431c7df6)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:27.352521 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:27.402427 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20cf8769967d4169be2a2a82431c7df6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.409571 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20cf8769967d4169be2a2a82431c7df6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.410251 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20cf8769967d4169be2a2a82431c7df6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.410866 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20cf8769967d4169be2a2a82431c7df6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.418347 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20cf8769967d4169be2a2a82431c7df6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.437673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1465/2944] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:37:27 2022] DELETE /identity/v3/users/20cf8769967d4169be2a2a82431c7df6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:37:27.612048 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:27.626206 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2656c-8183-40fd-8b2c-be79730e7621 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:27.687374 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:27.688005 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:27.688290 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:27.688565 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] PATH_INFO: `/v3/projects/63e7ee5c71674c23aca48909b417a52a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:27.689716 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e7ee5c71674c23aca48909b417a52a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:27.691659 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:27.737202 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e7ee5c71674c23aca48909b417a52a', 'request_id': 'req-06dc912b-e0c5-496d-9206-2c367e648611'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.743789 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e7ee5c71674c23aca48909b417a52a', 'request_id': 'req-06dc912b-e0c5-496d-9206-2c367e648611'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.750533 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e7ee5c71674c23aca48909b417a52a', 'request_id': 'req-06dc912b-e0c5-496d-9206-2c367e648611'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.762047 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] The token cache is being invalidate because project 63e7ee5c71674c23aca48909b417a52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:37:27.767332 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.768218 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.769755 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1480/2945] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:37:27 2022] DELETE /identity/v3/projects/63e7ee5c71674c23aca48909b417a52a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:37:27.774320 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:27.775784 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:27.789417 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b229eb-0ea8-44f4-b45a-1e13a707c5bd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:27.824129 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:37:27.850350 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:27.850976 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:27.851288 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:27.851567 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] PATH_INFO: `/v3/users/840fc526a7fa4831835b4757e8ee1e38` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:27.852690 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=840fc526a7fa4831835b4757e8ee1e38)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:27.854673 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:27.904059 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840fc526a7fa4831835b4757e8ee1e38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.911483 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840fc526a7fa4831835b4757e8ee1e38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.912154 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840fc526a7fa4831835b4757e8ee1e38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.912769 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840fc526a7fa4831835b4757e8ee1e38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.919319 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840fc526a7fa4831835b4757e8ee1e38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:27.937579 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1466/2946] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 30 16:37:27 2022] DELETE /identity/v3/users/840fc526a7fa4831835b4757e8ee1e38 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:37:28.110823 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:37:28.112612 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:37:28.125993 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc912b-e0c5-496d-9206-2c367e648611 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:37:28.172529 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:37:28.191156 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b24720c8535f4d608668a1e3b4a4419c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82db1e68be334ddf84a8e69d95cda33e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:37:28.191871 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:37:28.192111 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:37:28.192384 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] PATH_INFO: `/v3/projects/8bc96483ba614f5a8b1b71f53292a240` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:37:28.193508 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc96483ba614f5a8b1b71f53292a240)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:37:28.195453 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:37:28.243657 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc96483ba614f5a8b1b71f53292a240', 'request_id': 'req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:28.250982 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc96483ba614f5a8b1b71f53292a240', 'request_id': 'req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:28.258521 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc96483ba614f5a8b1b71f53292a240', 'request_id': 'req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:28.269406 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] The token cache is being invalidate because project 8bc96483ba614f5a8b1b71f53292a240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 16:37:28.269702 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:28.270381 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77e21a30-b5f3-47cb-934e-a6986f9dd6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 16:37:28.271863 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1481/2947] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 30 16:37:28 2022] DELETE /identity/v3/projects/8bc96483ba614f5a8b1b71f53292a240 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 16:37:37.933829 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:37:38.266430 dv-cinder-852208-13-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 16:41:37.647471 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 16:41:37.648955 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:41:37.720084 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4429c-6942-466a-b20b-be1855607bf9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 16:41:37.780386 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c81e08e950648be89483f26f546a7de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2f6f22dea344c4b87d4ee93a282f8ef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 16:41:37.780997 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 16:41:37.781283 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 16:41:37.781584 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 16:41:37.783390 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 16:41:37.795367 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 16:41:37.796952 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf56740-db12-48fa-8cfe-713a8a7c4a50 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 16:41:37.806864 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1467/2948] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 30 16:41:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 16:41:47.802699 dv-cinder-852208-13-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=58 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}